Could this be the future of Elgg data access and ui rendering?

https://youtu.be/WiO1f6h15c8

I found it incredible that this team in from Netflix has devised an efficient way to let the view layer of an app pretend that the entire backend of data is on the device. It has great caching properties, does not suffer from data duplication, etc. I was really inspired watching this today.

This, of all the strategies I've seen, comes closest to what I think Elgg needs, given how pluggable and extendable we want to be. If we can let the view layer decide what data it needs from a simple json-serializable graph structure, then we don't need to coordinate with controllers or components to marshall just the right data down to the client.

Watch the video. Let me know if you see the same implications for Elgg. I really want to try something using this approach soon. One of the things that I ran into very quickly with the Angular work was the need to tightly couple the views to the http apis that we're available. Falcor seems like it solves that problem. It's really too bad it isn't open-sourced yet.

Either way if we went this route I suspect we'd have to write a php implementation for the server.

Some things that aren't addressed that I'd want to research more:
* how to deal with sorting/filtering collections
* how to pin data offline
* how to do "create" actions
* how to handle variations in status for resources (e.g. distinguishing between deleted vs doesn't exist vs permission denied vs etc.)

All in all there's a lot missing from this story still but it seems like the foundation is so incredibly solid and helpful that it's worth investing more time figuring these things out...

  • This looks similar to Facebook's Relay/GraphQL with the same goals. There's a bit more duplication in Relay, as the template has to have a separate mechanism for reporting its data needs and what shape it wants it in (but the duplication is at least in the same file), and the public presentations didn't seem to go into client-side caching.

    There's not enough info on either to say which is better, but if the JS community can standardize enough of it this it could be a big shift in how web apps work.

  • The idea of making your whole data graph accessible in a single, json-like way is great... Firebase does something similar.

    Maybe it's a bandwagon but this seems like it really helps simplify things...

  • As for the present of Elgg data access, I think documenting what we've had in place for years is the minimum we should provide for 2.0.

    I'm going to make an effort to add a less painful way of setting up a JSON-responding endpoint that auto-wraps system messages like elgg.action. We can't radically change Elgg's data strategy for 2.0 but we can do what we should've done for 1.9.

  • Cool, please also take a look at firebase, it provides data synchronisation when not being online and still need to act as if being online.

  • I have seen firebase and think it's quite cool regarding the sync and offline stuff.

    Unfortunately it is a service and we probably wouldn't want to tie Elgg to a service that you cannot self host. Falcore has the benefit of being more like an open source protocol + libraries so we could pull in what we need, potentially.

  • @evan, I agree. I saw the drawback of that too.

  • I went through and made up a falcore-style api for accessing some data in the elgg graph. It's not exhaustive or anything but was a fun exploration of how things might work. The thing I like is that all the data access is very uniform: Everything is an object or a property and if you want to get to related objects you just walk the graph in a very easy-to-learn fashion. There's no querying/filtering/manipulating. All of that complexity is hidden away and you get to just focus on rendering the data in whatever format you like.

    https://docs.google.com/spreadsheets/d/1BE3vUejIudlUy3Kz55_lIQjpAKaHvEYiI2Q9sNMObSA/edit#gid=0

    Also, I've done some more research and come up with answers to some of my prior questions:

    How to deal with filtering/sorting collections?

    The filtering/sorting would happen behind the scenes of the falcore router using a lower-level API of your DB management system. To those accessing the falcor API it just looks like walking the graph:

    $root->me->blogs->search->{"Elgg"}->orderBy->recent

    This would presumably return a list of the logged in users blog's that match the search term "Elgg", most recently published first. But you have to explicitly wire up the router to do that -- it doesn't inherently understand "orderBy" or "search" keys as being special in any way. I suppose Elgg could provide an abstraction on top of falcore that does understand those keys as special, but it's not required by our use of Falcor by any means.

    How to do "create" actions? (i.e. actions with side effects)

    Falcor allows you to define RPCs in addition to data for those cases where setting a property on some object doesn't cut it. For example:

    $root->blogs->post(['title' => 'How to write a blog...', 'body' => 'This is the bestest way...']);

    We could come up with a scheme where the only rpcs we ever define are "delete()" or "post()", and that would feel more like a familiar HTTP API, but we're not restricted to that.

     

  • I finally got around to watch that video. Better late than never, I guess. :)

    Yes, very interesting stuff! It would be cool to build something like this for Elgg.

  • They finally open-sourced the code, too!

    https://github.com/netflix/falcor

    I'm working on an Falcor/Angular2 integration ATM. Feels very promising so far.

    One rub I sense is that porting falcor to PHP will be interesting given that PHP is typically sync/blocking whereas JS is all about async. There is a way to get async PHP but it's by no means the default or common configuration...

  • Fortunately there is supposedly a port already in progress, so it's not as if we'd be on the hook for it.

    https://twitter.com/falcorjs/status/596416483602731008

Feedback and Planning

Feedback and Planning

Discussions about the past, present, and future of Elgg and this community site.