LibreOffice - peer2peer collaboration bits

[mmeeks]

Hi Simon,

I’m excited about extending LibreOffice to include peer-to-peer collaboration:

@webmink

Sure. In the proposal … I suggest we add peer-to-peer (serverless) collaboration directly into the desktop LibreOffice, mostly by leveraging existing code. … It’s important that TDF’s investment in LibreOffice – which donors expect – serves our mission by bringing collaboration capabilities to ordinary users rather than restricting it to those capable of running servers or willing to become clients of a service provider.

This seems a great direction to me; quite a chunk of the feedback we get around CODE/COOL is - what about peer-to-peer PC application collaboration? (following a similar pattern to the competition) - people still love the desktop client. My expectation is that integrating this sort of functionality - particularly if it could be extended to collaborate between COOL, Mobile & web versions as well as between desktop/LibreOffices would be a great feature for the whole community.

I’ve a number of technical thoughts around how to achieve this, and Collabora is part of a project proposed around a prototype in this area that could be accelerated here.

There are a number of interesting technical problems: how do we do the protocol/API ‘cut’ here to make this work reliably while being deliverable this century (ie, without the need to completely re-factor the whole core). My preferred approach would be to make the ‘cut’ around the smallest & simplest APIs we have: keyboard & mouse input. We could/should extend that via Caolan’s dialog welding & Szymon’s JSDialog protocol in core which was done for COOL to provide richer dialog semantics, as well as re-using the multi-window/view model approach use in COOL. This would have the benefit of being an extremely complementary and achievable technical approach that can be incrementally deepend as we improve the core to include more interesting operations/transforms as & when. We should probably also be extending the JSDialog/weld approach locally to implement proper native widgets support for Windows/Mac/Android?/iOS and better look/a11y too - though I guess we could wrap VCL widgets initially.

Of course, this will need to be discussed and planned properly with the ESC.

Anyhow - I’m encouraged to see a positive idea here that is constructive, collaborates with and leverages existing ecosystem investment around LibreOffice Technology, significantly enhances desktop/LibreOffice for the future, potentially allows new chat companies to get involved in collaboration channel setup and interested in LibreOffice, and so on. Seems like a direction we’d want to support at Collabora, and I’m excited by the prospect of a new board term and focus with the chance to working together constructively.

I look forward to digging into the technical details, at FOSDEM or sooner if people are interested.

Thanks,

Michael.

[regisperdreau]

Hi,

Wow !
Sounds marvelous
It’ a great idea that many users expect (as Calc color filter was …). We speak about with users at OSXP and other exhibition with La Mouette.
Just some wishes :
The best approach should be :

• easy to learn
• easy to document
• easy to improve
• easy to debug
• easy to fix in a reasonable time
  by design !

(I presume this functionality as described has a high potential to become a Heath Robinson engine that nobody can learn, can develop and maintain outside first developper team). Someone has to support it, one day…

Best,

http://transpressnz.blogspot.com/2013/11/the-heath-robinson-engine.html

Régis

[htietze]

Some kind of communication protocol for office clients sounds like a great idea.

I have a few questions:

• How to connect two or more individuals? It requires routing, and given that users not want to fiddle around with ports and firewalls neither to share IP addresses I presume this requires a server. What could be achievable on TDF infrastructure?
• Remote access is typically laggy even on locally connected machines. And it’s not granted that on a certain position in the UI the same command would be executed on click. Isn’t it better to share UNO commands and parameters?
• I’m not convinced that input would be synchron. How do we solve the situation when one participant enters text and another deletes the same paragraph? (Would be only relevant if we update less frequently; by submitting every keystroke it should be clear who types; or we show the information “Foo is typing…” as known from chat programs).
• How much effort would this mean, and what milestones could be defined?
• Encryption and data integrity is key. But the experts surely have some protocol in mind.

[mmeeks]

Hi Heiko,

We talked these through in person at COOL-days, but for the benefit of the list:

Some kind of communication protocol for office clients sounds like a great idea.

Great, glad you like it =)

I have a few questions:

• How to connect two or more individuals? It requires routing, and given that users not want to fiddle around with ports and firewalls neither to share IP addresses I presume this requires a server. What could be achievable on TDF infrastructure?

TBH - I’m not hyper-interested in the channel. We just need an ordered event delivery service - I guess something like this could be hosted by TDF without significant issues; the CryptPad folk do that nicely. Some trivial un-encrypted event echoing helper would be fine for development.

• Remote access is typically laggy even on locally connected machines.

Are you sure remote access on a LAN is laggy? I’m far from convinced by that. To scroll and navigate in the document shouldn’t require any round-trips at all - the model would be a local copy.

And it’s not granted that on a certain position in the UI the same command would be executed on click. Isn’t it better to share UNO commands and parameters?

I’m not optimistic about any approach that sounds like using the macro recorder to achieve coherent documents As I said above - the interesting part is: “how do we do the protocol/API ‘cut’ here to make this work reliably while being deliverable this century (ie, without the need to completely re-factor the whole core).” and then also: “My preferred approach would be to make the ‘cut’ around the smallest & simplest APIs we have: keyboard & mouse input.” Possibly some UNO commands in some context are not a terrible idea there - but they are -far- more complex than key-stroke input - and already making mouse-input / cursor movement resolution / font-set, etc. independent is not entirely trivial.

• I’m not convinced that input would be synchron. How do we solve the situation when one participant enters text and another deletes the same paragraph? (Would be only relevant if we update less frequently; by submitting every keystroke it should be clear who types; or we show the information “Foo is typing…” as known from chat programs).

My approach solves that problem by re-using all the existing logic (already in the core LibreOffice technology) that is also used and tested by COOL for these cases. This works well already - you can test that out locally by opening two windows (Window->New Window) and alt-tabbing between them while trying to delete the other windows’ cursor etc.

• How much effort would this mean, and what milestones could be defined?

Good questions for a project manager to ask and nail down, as well as helping to do the vast work of consensus and trust building to make it possible for people to imagine doing something constructive together here.

• Encryption and data integrity is key. But the experts surely have some protocol in mind.

Of course CryptPad provides a very nice infrastructure for doing this - I would really suggest not re-inventing the wheel particularly in a security critical area; we have enough fun problems to solve in LibreOffice =) We should re-use a high quality foundation from others I believe - but there is no need to be firmly tied to any one channel.

Hope that helps - and of course it’s my 2 cents,

Regards,

Michael.

[htietze]

@mmeeks

We talked these through in person at COOL-days…

Yes, for transparency and perhaps more opinions here. Hopefully the legal questions in the other thread will be sorted out soon.

[EyalRozenberg]

@htietze

How to connect two or more individuals? It requires routing

I would opt for a simple protocol which does not take on problems more complex than it has to, at least for a preliminary implementation. Specifically - the Internet routes for us.

@htietze

given that users not want to fiddle around with ports and firewalls neither to share IP addresses I presume this requires a server.

But then it would not really be P2P, would it?

Anyway - a server would not necessarily be required. That is, P2P connection will happen when two users want to connect; but in our case, they have already “connected” in some other way to agree on making the P2P connection. So I suggest, in light of my previous point, that we assume that the two (or more) users have another, independent means of communication over which they can send some data for bootstrapping the LibreOffice P2P. And this could be made easy, UI-wise, so that the user just needs to press a copy button, and paste some string so that the other user can see it. The other user copies the string and pastes it into an appropriate area in their own running LO instance. Then the connection is set up.

What kind of data? Basically, I assume that would be a tuple of (IP, port number, public key). I will admit that this doesn’t cater to the case of two firewalled users; that’s a situation I’m not experienced enough in handling, but I do know there are many approaches (Wikipedia) to handling it. Some may require a third-party “switching server”, some may not. But such a server can probably be very minimal and hopefully not even aware of what protocol it’s being used to allow connections for.

@htietze

What could be achievable on TDF infrastructure?

Given what I’ve said above - let’s try to make this completely independent of TDF infrastructure. Either with no switching-server at all or with something minimal that hopefully might not even need TDF continuously maintaining a server. Note that maintenance by us also has privacy implications, much more so than third-party-less P2P.

@htietze

Isn’t it better to share UNO commands and parameters?

Mmm… maybe… but - what about showing the other party’s cursor and mouse movements? You can’t do that with UNO commands.

@htietze

How do we solve the situation when one participant enters text and another deletes the same paragraph?

It doesn’t have to be a great solution, as long as it is consistent. i.e. if users know that two people on a laggy connection editing the same sentence is likely to get them making changes in wrong positions etc., they will naturally limit the extent to which they do this - like we know from Etherpad. Consistency of behavior and “principle of least astonishment” would be more important than perfect coordination/synchronization of inputs.

@htietze

Encryption and data integrity is key.

Perhaps TLS if it’s a TCP-based protocol, and DTLS if it’s UDP? Using the exchanged public keys I mentioned before?

PS - I’ve mostly focused on the case of two users. More users makes things trickier.

[thb]

@htietze @EyalRozenberg @mmeeks I took the liberty to move this part of the discussion over to the developer list. I believe technical details are better discussed over there, hope that’s fine for you!