Elgg at a crossroads

I'm excited about the upcoming Hackathon (Sept 23, 24), but this temporary burst can't make up for the fact that core dev contributions are down and have been for a long time.

This isn't anyone's fault. To an extent, this is normal cyclic behavior as people get busy with other things, but I rather think it's that Elgg is no longer exciting or fun enough to maintain developer interest.

Sure, we've made huge improvements in DX and performance, and 2.x provides a lot of value, but the product remains highly tethered to trends, technologies, and architectural decisions made 8 years ago. We have some loud voices in our community telling us that Elgg should change even more slowly (or not at all), and I've personally over-romanticized the idea of extreme backwards compatibility, but if we continue this path I think developer interest will keep dropping off.

Personally I really enjoy working with Elgg's contributors and don't want to stop, but for my career's sake I can't focus on reworking 2008-era APIs much longer. I considered quitting in 2014 and didn't out of inertia and because family health problems made it comforting work to bury myself in.

What can we do?

We could do nothing. This is just my perspective and I could be wrong or it could really not matter. There's nothing wrong with stable software that nudges along slowly. My prediction is that high maintenance chores will start to go by the wayside. Elgg 3.0.x will get dozens of releases and plugins will thrive as they did on 1.8. It's not a bad outcome for lots of people, but I'd be bummed.

In the short term, Ismayil and I should stop stalling new development in order to wait for a 3 member consensus. If a PR on 2.x/master gets a thumbs up and no one expresses concern or a desire to review for weeks, it goes in. Those branches give plenty of opportunity to revert or fix concerns that come up later. When reviewing involvement goes up, we can scale the need for consensus accordingly.

Businesses relying on Elgg could offer developer time. That won't help if devs really don't want to work on it though. And IMO business owners are under no moral obligation to contribute, but it would of course help.

We could seek out and destroy any low-return hassles that make working on Elgg less fun:

  • Commit messages are a constant headache, but may be worth it for our awesome changelog.
  • LTS releases: if so many sites want to run 1.12, one of their developers should put in the work of backporting fixes to it; merging this stuff into 3 other branches is a headache. Maybe the foundation can pay someone to make these releases occasionally.
  • Community spam: Maybe you can't register until you to contribute some sort of content; then we can just delete several thousand spam profiles.
  • RST is still !@#$. I'm continuously looking up how to do basic things and even the docs about RST are bad. Markdown is inferior tech with miles ahead UX. Really I want to do like Drupal and build more docs from source code.
  • ??

We should identify roadblocks that make more radical changes seem impossible. It was really disheartening to turn away a good 3rd party effort at moving to UTF8MB4 columns. Schema changes in general are almost unimaginable due to the expectation that Elgg run its own upgrades.

Personally I would love to--at least temporarily--set aside the requirements of upgrade paths and BC and start over.

  • How can Elgg be many magnitudes faster, not a few ms.?
  • How can it have the real-time functionality users now expect?
  • Are we sacrificing anything by giving plugins too much power?
  • What Elgg-specific dev concepts are really cool or not worth the overhead for new devs to learn?

In general I want Elgg to take a great leap forward to become something I'd be proud to put my name on, not filled with things I have to apologize about.

  • @ura, I agree with you but:

    simply because (imo) they focus on the superficial and UX experience primarily which, rightly or wrongly, gets people inspired

    this statement to me is a bright example of how much usually devs consider users, non-devs as "simply" focused on superficial aspects, maybe simple-brains.

    How many car drivers really know how an engine works? And, mostly, how many car drivers are willing to learn how to build, mantain, repair their cars?

    Are they all stupid? No, lack of time, different interests, no skills. But they still want to drive a car and the process leading to a choice is getting more and more complex due to increasingly higher exposure to ads of every kind and through every media.

    Now, having said that I have the higher respect for all of you who are sharing your skills for the sake of knowledge and continuously improving....up to kind of feeling frustrated like Steve and Ismayil here because willing to do better things, please consider the following: how would Elgg-based sites be without the following non-core features (I'm not even mentioning minor ones), from user perspective?

    Photo, video, share, wall, html notifications, profile manager, advanced groups tools, events, comment tracker, maps, feedback, polls, antispam, FAQ.

    Many months ago I asked what the main problem was with Elgg development, money, number of devs involved or else...the reply was few devs involved...I'm doing my best to bring some in...last event helped in making Elgg a bit more known here...some are slowly starting to like it...fingers crossed, heads up! :)

  • how would Elgg-based sites be without the following non-core features (I'm not even mentioning minor ones), from user perspective?

    Photo, video, share, wall, html notifications, profile manager, advanced groups tools, events, comment tracker, maps, feedback, polls, antispam, FAQ.

    Elgg only gives us this:

    Wall, HTML Notifications.

    Let's be real for a second: The majority of users want something that just "Works". They don't want to look for solutions or spend time searching for the right configuration. Users want something that will have everything they need. There's no photo gallery plugin out of the box. Not even a link preview for "The Wire". Even the "Profile Message Board" is extremely basic, you cannot share pictures, files, or even a link with a preview. I know which plugins I will need for each one of those features, but most users don't want to spend time looking, that's why I feel they ditch elgg for other platforms, such as SocialEngine.

    Can we at least consider the idea of creating "Core Council" and "Elgg Community Council" to decide a clear strategy? Elgg core developers need help. We can help.

  • Actually I meant Ismayil's Wall+extended tabs we funded and shared with community. Unfortunately Elgg's (river? wire?) is not really enough for users looking for a modern interface/experience.

    Html notifications...well, I have Coldtrick's installed, is core now not needing it anymore (I'm still on 1.9)?

    If you guys think a non-dev could help...I would love to contribute in giving feedbacks ("council"?) which now I'm more confident on having myself received many real-users feedbacks in last couple of years.

    Cheers and always up with Elgg!


  • BTW, just to pour some more positive attitude: we're now talking about how to improve Elgg and Community but current situation (2.2) is waaay better than 1.7+ so...let's think about how much great work has been done and then how to improve :)

  • I think the main job of the core is to make ithings possible, to provide API for plugins to implement features and needs that vary from site to site. We can't stock everything into core - on the contrary there is too much overhead with the bundled plugins. Do we really need blogs and pages together? Do we really need the wire and the inferior messageboard? Why do we need bookmarks - can't people jusy use their browser bookmarks or write a blog post if they want to share an article?
    I agree that all these post types are ambiguous - we should instead develop a rich post interface that reflects the needs of modern social networks - ability to attach content and rich media, out of the box mentions (users, groups and other local "bookmarks") without absolute URLs (ECML to the rescue).
    We should focus on making admin experience better: there should be a visual toggle for all config values. It should be easier to modify site/footer menus and link to static pages (get rid of expages in favour of anypage and improve it).
    Instead of bundling plugins we could create recomended composer json variations in the starter project that would build a project for specific use cases.

  • If this is not the appropriate place, I apologize and let me know I'll move it or whatever.

    Not to continue beating a dead horse or continue bashing, but I am going back to reread all of these comments made here after I write this, but I am so frustrated (been so for almost a year now) that I felt a compulsion to immediately vent.  I apologize in advance, but this hit home. BTW, I am glad this discussion has been started because I have a very bitter taste in my mouth from my experience.  I am not sure of the exact problem in the elgg system causing my issues, but as a user, yes, we just want something that works and NOT AT A CRAWL to a page load of 4-5 minutes per Event Calendar query (before you start thinking its bad code, it was developed by a programmer here with a lot of elgg experience and plugins). Could it be the database structure? Maybe...in my limited knowledge of the elgg database, I am still inclined to think so.

    My project is of an Event Calendar with copious data and mapping.

    I was a very excited user because of this project, hired someone from the elgg group to customize his event calendar for me back when it was I think 1.18 or something.  I just wanted something to work and it did for small amounts of data.  But after spending over $17,000, through 2 programmers ($6000) and 1 mobile app company ($11000), I still DO NOT have a functioning site.  It has now been 3 years since I started.

    The version updates and lack of backwards compatibility is part of my frustration.  I am not able to keep the plugin up to version updates. UGH. But I surrendered to stopping at that version (because of the event calendar plugin in particular), which I didn't mind doing, because I just wanted something that worked.  It did work at first, with small amounts of data.  I didn't expect for the event input volume to gravely affect the download time of its pages.  More UGH. I am at a loss of where to turn.  My site(s), after several iterations, and plugin minimization, reinstalls, the event calendar pages take literally 4-5 minutes to load per request even just out of the box, after about 2 weeks of constant event inputting -- and the list of event being queried and mapped on the page is just 20 at a time! Even more UGH. Plus, new versions do not work with the event calendar plugin I had built, so do not know if it would improve anything.

    That prompted me to hire that company to do a mobile app to develop something that could work with the elgg database, workaround it's super sluggish front end, and instead going through the backend developing an apk to handle the data directly.  It is faster, but still slow.  I had to stop on the mobile app because I didn't want to spend any more money on something until I could figure out what is the problem, fix it or whatever.  I don't think I can use the elgg system except MAYBE as a core collection tool for event data, but it is still sluggish. 

    I guess this is more of a rant than usable input, I am by no means an elgg expert or elgg programmer, but I felt it was at least some feedback from a user that just want something to work. As rjcalifornia said, we just need something that works and isn't outdated like mine is after just a couple years because customized plugins fail to work and it is costly to maintain them every time there is an enhancement.

    So I guess I am in the market to discuss yet another hire from here to help me.  If interested shoot me a message.  I already went through Freelancer and that other one, but I guess it wouldn't hurt to try looking for a programmer here.  Perhaps over there they aren't as familiar with elgg (particularly the database) as they say they are.

    Peace, Kepa

  • @Kepa I'm sorry about your experience. We're happy to help you--or your developer--via the community, but let's leave it out of this thread. Post as much detail as possible to a new topic in the Performance group. Particularly helpful would be access to MySQL's slow query log. How long do requests take for pages with little overhead, like https://example.org/admin/statistics/server ? What's the Elgg version? Is 4-5 minutes under no load or with lots of users online? Again, post these answers elsewhere :) -- Edit: I'll just add that Elgg's data model is fine for some things, but generally you'll need to use custom tables to get better performance, either to denormalize, cache expensive operations, or just match the data needs better.

  • @Kepa could it be that you asked me once about your problem with the Event Calendar plugin? It rings some bells. My answer probably included that I don't do custom works as I only code as a hobby. But I think I also made some guessing about the reason for the long page loads (which seemed to happen only with your modified version but not with the unmodified version of Event Calendar). As far as I remember your problem you are using a modified version of the Event Calendar plugin with some Google Maps feature added. It worked fine with only a few events but no longer with many events. I think the problem is that there's a query made to the Google Maps server for every existing event on every Event Calendar page load. And this will simply not work in any way like this. The number of queries must be reduced significantely, either by only making a query when viewing a single event entry in full view of at least to limit the queries to events currently displayed in the event list of the current page.

  • for anyone who is thinking about UX in social sites, this new video from devtips is a detailed look at site design using the evolution of facebook as an example: https://www.youtube.com/watch?v=UmKD1ciQg2A

  • I am not able to keep the plugin up to version updates. UGH. But I surrendered to stopping at that version

    This. I know there are several elgg powered sites/apps that won't upgrade their elgg version. We have an SaaS app and we decided not to upgrade versions anymore. At one point we were keeping up with each release, but we were mismanaging development time when we focused on upgrading every time a new version was released.


    Actually I meant Ismayil's Wall+extended tabs we funded and shared with community. Unfortunately Elgg's (river? wire?) is not really enough for users looking for a modern interface/experience.

    @Michele Sorry, I think I didn't explain myself. The problem is that none of the features of Ismayil's Wall have been implemented in the core version, which is a shame. We are falling behind times.




This discussion is closed.

This discussion is closed and is not accepting new comments.

Feedback and Planning

Feedback and Planning

Discussions about the past, present, and future of Elgg and this community site.