Doesn's even a single user instance on "community" version shake itself apart and needs uh, rather a lot of resources to run?
This feels like a cruel joke
@element Yeah, the funding situation sounds awful, and hope it'll help you out. Especially given the sheer difference in what it costs to run the python workers vs the Pro rust ones, that probably eases up the hardware costs a lot for you. And well, anyone who gets to run this.
Do hope for the day when we can have groups of friends and communities not using Discord, but using a nice little Matrix server running just on an RPi without.. extra tuning or crashing ^^' Not sure it's currently viable with the python implementation, beyond like, single digit user count
@element glad to hear there's progress on addressing that, hooray ^^
And I imagine "join a large room" is a pretty common use case for single user instances, especially given the amount of various open source communities that use matrix as primary contact point and discussion space
đšNation-scale Matrix deployments will fail if built on the community version of Synapse.
The community version of Synapse is not designed or intended for use by commercial Matrix hosting providers to serve huge nation-scale deployments. Just as a suspension bridge has a weight limit and will collapse if you exceed it - the same goes for community Synapse.
Deployments supporting millions of users need Synapse Pro.
https://element.io/blog/scaling-to-millions-of-users-requires-synapse-pro/
@element As a tax payer I want my government to use open source. Since Synapse Pro isn't, it's simply not usable for such cases.
Thank you for detailing the need for doing the same development to the open source Synapse as you have already done on Synapse Pro. I hope you do agree, it would be a pity to have to fork.
@troed We'd also like everything to be open source. But the reality is that even with AGPL, we're still stuck in the pattern that enormous deployments use FOSS Synapse without contributing to its dev/maintenance costs.
Every time an opportunity that we're counting on to fund the team turns around and says "oh, FOSS Synapse is good enough, and we can use it for free, bye" we find ourselves needing to provide a very concrete reason not to freeride - hence Synapse Pro.
@element What is the motivation to maintain two different codebases (Python and Rust) when the community version is not profitable? Isn't it cheaper to have one version for both paying and non-paying users?
@element Itâs ok, not everyone can be good at engineering! The important thing is that you tried and did your best
@element so reading through the article and through the replies, some questions come up:
- why us scaling only important for big deployments? I don't have the money to throw at hosting my own home server for ~20 people with how many resources that already needs 1/2
Pretty sad Element is fucking themselves over. The death of Dendrite makes more sense now. There are a lot of other Matrix clients, but very few re-implementations of the server (none?). The blog post talks about government service. Congratulations Element, you're now selling yourself to wasteful entities that only by overpriced shit and screw over the population.
It's like they're managing to piss off all communities and forcing themselves to the mediocre middle.
Glad to know you're open source product is literally designed to fail! Great marking in your blog post.
You will never be Slack, and now you'll also never be XMPP.
@element I really do hope that this post is a marketing disaster and those responsible for it must be fired.
If that's not the case and gating performance improvements behind the paywall is actually their strategy, then I should say that this behavior hinders the adoption of Matrix for people as they see it as very slow, and not everyone would be ready to run their own server as the most feature-complete server Synapse is working slowly and requires many resources by design.
I guess we should learn that we never should trust open-core companies. I think that paid support, hosting and third-party integrations are ok, but gating essential features must not be acceptable.
"#NationScale" is a funny #bullshit #marketing term.
It is not clear, if it means IN or VA, but it sounds so powerful.
@element Do you know of any successful open source businesses that have used this particular method?
"How to make money from FOSS" is an age old problem - but "the free version has a sucky implementation" is, afaik, not the most successful one.
@element Would you be able to name a price if one such "nation-scale deployment" wanted to pay you to switch Synapse Pro to AGPL and maintain it as such for, say, 5 years?
@michal FOSS Synapse is the primary project, and is where all new features and maintenance happens. However, it doesn't fund itself, so the cost of also maintaining a paid Rust port of it is worth it if it lets us keep developing and releasing normal Synapse to everyone as FOSS.
@element - I thought being able to scale federation workers is very important because ideally everyone can run their own homeserver and then you suddenly need a lot of federation workers if the ecosystem worked like it's supposed to (going back to the first point)
- why the hell is @matrix running a proprietary implementation and why is everyone funneled to that intransparent implementation through the UX on matrix.org and clients like Element? 2/2
@WerySkok the reason synapse is slow for small servers is not because the workers are slow, but because of state resolution, state storage, room joins and full-mesh federation being slow. The point of Synapse Pro is to raise $ to fund optimisation work to fix that (and other Matrix work) as FOSS.
@troed we're not remotely trying to do "the free version has a sucky implementation". FOSS Synapse will keep improving, including perf (but not scalability) work. The model is more like MySQL Enterprise (back int he day). "If you want a massive cluster with a guaranteed SLA, here's our paid clustering product".
@nemobis the funding gap we're try to plug for Synapse and associated Matrix dev is very roughly around $5M/y. So over 5 years, that'd be $25M.
@sunweaver @troed the problem is mainly that system integrators turn up saying they can provide that support and sla by using the foss project, and then try to save costs by not working with us as the upstream - meaning the upstream isnât funded, undermining the underlying project, and increasing the chances of the delivery going wrong without expert support (and with a weakened upstream).
@element Do I understand correctly that those optimisations will be implemented in FOSS version if enough amount of licences will be sold? What's the roadmap for that?
@element Right, so let me make a quick rewrite:
"Scaling is hard!
We're seeing many successful Matrix deployments using Synapse for which we are extremely proud and happy. We're also seeing some running into scaling problems as they approach hundreds of thousands of users.
Element is here to help. We're the biggest contributor to the open source Synapse community, and as such we have unparalleled expertise when it comes to optimizing very large deployments. With our performance libraries, that are specifically crafted for Synapse deployments that can reach millions of users, we can get the performance of massive - and costly - server solutions down to something that's much more manageable.
If this is a good fit for your deployment plans, please reach out!"
@element Thanks! A very reasonable price if you ask me. Italy is planning to spend *billions* to supposedly secure their insecure military messaging at the network layer.
@davebloggt @element @matrix @robin ties directly back into my first and second question. Also matrix.org is still away from "nation scale" so shouldn't it be fine with running regular synapse?
@sunweaver @troed https://www.heise.de/news/Probleme-mit-Open-Source-Videokonferenz-Tool-Hessen-fuehrt-kurzfristig-Webex-ein-10217839.html is one concrete example, which contributed directly to layoffs at Element in 2022 and 2023 as mentioned in https://matrix.org/blog/2022/12/25/the-matrix-holiday-update-2022/. Synapse Pro is an attempt at us giving SIs a much more valuable reason to work with us as the upstream rather than trying to cut us out.
@WerySkok those optimisations will be implemented in FOSS whatever, if thereâs $ to fund it. the roadmap depends entirely on if thereâs $, either from Synapse Pro or elsewhere, and is the only reason it hasnât happened already.
@troed that is a great pitch - and is fairly close to the original Synapse Pro announcement from https://element.io/blog/synapse-pro-slashes-costs-for-running-nation-scale-matrix-deployments/. The only problem is that then a bunch of big deployments said "that's nice, we'll risk it anyway". Hence the harsher "seriously, you will fail if you try to wing it" wording here.
@nemobis yup, and be vendor-locked to SpaceX in doing so: https://www.bloomberg.com/news/articles/2025-01-05/italy-plans-1-5-billion-spacex-telecom-security-services-deal.
Another solution here could be if governments decided to actually support FOSS by funding its maintenance & dev by subscribing to the paid product, rather than freeriding. ZenDiS in Germany is a great example of this. But unfortunately, they're almost unique in this, hence Synapse Pro.
@famfo @davebloggt @matrix @robin matrix.org has around 4M MAU which is definitely 'nation scale'. The use of Syn Pro there is to save cost for the Foundation tho.
However: we should have made it clearer that algorithmic optimisations for small servers (i.e. being able to federate more efficiently) will absolutely keep landing in FOSS Synapse. Working around those problems with faster workers is a bandaid. Whereas if you had a massive deployment, faster workers is the right soln.
@element Ok so now I'll need to preface the next part: I am a former Head of SaaS that built a commercial product on top of a FOSS stack. We took the decision, multiple times, which parts to use as-is, which to pay for commercial support and what that cost/benefit looked like.
I'm sorry, you cannot scare potential customers into choosing your Pro product* over the FOSS offering. If they have the budget and their mission is clear, they will pay. If they don't, they'll need to miss their internal targets _before_ they're able to acquire that budget.
The only thing you can do is to wish such a customer good luck, and then sneakily have your sales personnel check in every now and then. If your Pro product is cheaper for their use case, they will come back and pay you once they've figured out they weren't as smart as they hoped.
Now, I asterisked "your Pro product" - because the big difference between your original product announcement and my rewrite is how you phrase Synapse Pro as _a different product_ and not just as paid consultancy/contract to optimize a Synapse deployment. That way you're basically telling your customer they have a one time choice between Pro and Community, and if they choose Community there's no easy "come help us!" path forwards once they run into scaling issues.
I do understand from this communication that things are tight though, and I truly hope that the EU, or individual governments, can finally understand that this is the area in which they should spend money. But, back the original post. As a dev, and former head of blabla, that post really came out as "we made the FOSS version sucky and put the good code into our proprietary product".
fwiw
@element @davebloggt @matrix @robin I'm going to be honest, "this is our proprietary bandaid for the existing code being slow" isn't the response I would've hoped for but good to hear that there is at least the intend to bring a proper solution downstream.
PS: the matrix.to link is invalid in my client but I think I found the event you were referring to
@troed yup your point makes sense - especially as Synapse Pro very much *is* designed to be dropped into speed up FOSS Synapse when needed. Well, we can always do a 3rd blog post with spin to see how it lands... :)
@famfo @davebloggt @matrix @robin the point was more "faster workers are for huge commercial deployments. they're not the right solution for small server perf, where instead we should fix the actual protocol & FOSS to be faster". which should be good news.
@element @troed synapse pro is assuredly based on other open source software, potentially even existing community matrix protocol libraries like ruma
how are you ensuring that you yourself are not freeriding on these community projects? there is not much of a difference between selling server software while not giving back to open source devs and people selling your service without giving you money.
@alxlg @troed Ironically, we already were doing something like the Open Nitrate Model. https://element.io/server-suite is the proprietary product which lets organiations lock down how the FOSS products work. However, in practice that hasn't been enough.
And yes, we do contact the institutions and explain they need to support the upstream - but if they don't have budget (because why would they have budget to support something that's free-as-in-beer?) then it's a problem.
The Element web page clearly says:
"Element is an open source company."
And:
"Element is committed to open source"
No doubt, that Synapse Pro will be released under a #freeSoftware and #openSource license. I'm sure, we will see the repository on #MSGitHub, soon:
@debacle @patrickleavy @futo Yup, we're committed to open source - we're continuing to release Synapse and all of Element as FOSS. FOSS Synapse is still our primary server project. Element has always had a handful of proprietary products too (e.g. Element Matrix Services; Element Server Suite) to try to keep the lights on.
@element honestly, I'd consider in your place what you expect to achieve by that. Most public and corporate channels are not and won't be e2ee. So your pretty much only advantage over xmpp is becoming irrelevant. Performant wise you still are not competing even with xmpp - xmpp it's far superior in terms of speed (even nowadays mobiles). Maybe some power consumption is still concern in xmpp but doable. So, what value you provide here? I'm not sure. I used to be fan of matrix
@endofline all of our paying customers want E2EE without exception. Performancewise, XMPP has always been better than Matrix? The reason to use Matrix is if you want decentralised conversation history; E2EE-by-default; E2EE group VoIP/Video. Folks should use whichever fits their use case best.
@endofline Just in case anyone needs to know this (I meet a lot of XMPP-haters who last used it in year 2000 and think it never progressed) - #XMPP has a number of fairly usable end-to-end encryption protocols, including PGP and OMEMO. The latter is built on the same double ratchet algorithm as #Signal and #Matrix, and is supported by all major clients.
Sure, the UI/UX could be improved. It's FOSS, and doesn't have the double-edged sword of VC funding - patches/donations welcome!
@contrapunctus @endofline hate to do this but a thousand reply guy assholes are going to jump at your throat for posting this.. as an XMPP user, sadly, OMEMO is wholly inadequate for E2EE, I've even found that it's less of a hassle to just disable it rather than endure the key verification steps that are very different across clients.
XMPP should never be used for secure messaging where it is critical that no data should ever leak, neither should matrix.
EDIT: woe is me, i was actually replying to the fossbro reply guys. Adventurer who enters this thread beware, here be dragons
that's critical phrase imo:
> In my opinion, itâs not.
Well, xml messages are indeed antiquated but at least they do work. Matrix I'm not sure - it's never working for mobile story , conversations do work nicely. Regarding E2EE I'd need some details to see the problem: are there some metadata leaking or what? XMPP only real issue is adoption. One advantage XMPP has - it's already working server-wise and has multiple nicely working clients.
@SharpLimefox Please don't post that misinformation-riddled post. The issues are either misguided or already fixed.
Here's a response to it. Also includes a comment from the OMEMO author, which Soatok chose to delete from their blog.
https://www.moparisthebest.com/against-silos-signal/
If metadata protection is a concern, a self-hosted private #XMPP server makes a stronger case than any third-party service - especially over a target as prominent and tempting as #Signal.
@contrapunctus @SharpLimefox Yeah all of that has been disproved or resolved for a long time.
@bhhaskin @contrapunctus sorry, that's on me for thinking people could have reasonable discussions online about software
@SharpLimefox @contrapunctus instead of saying we are being unreasonable, explain which part isn't factual?
@bhhaskin @contrapunctus no because neither of us have a realistic need for this. You don't know me. I don't know you. Neither of us care enough beyond our egos. Why do you go pick arguments with people online instead of, idk, going outside to touch grass
@bhhaskin @contrapunctus i could engage further by asking "show me when the issues got fixed" but neither of us care and we'll both find points to nitpick about. This is the whole point of this dance.
@bhhaskin @contrapunctus XMPP can let you turn off encryption. There. That alone disqualifies any claim of its use as a secret messenger. Then there's OMEMO versions. Give me a whole list of clients and their versions. Find the one everyone uses in reality. Give me the CVEs they have. Then meditate on the time we both lost.
Have a good evening, sir.
@KyLeggiero all the pricing options come with Synapse Pro (other than Element Cloud hosted servers, which are lagging; they will get it in the end though); we should update the page to spell that out. And yes, the pricing is aimed at being sustainable rather than greedy.
@element Didn't you consider https://opensource.org/delayed-open-source-publication this license? Like community open source implementation 1-2 years behind. Open source up-to-date only for matrix protocol
@endofline we haven't ruled out FOSSing Synapse Pro in future (or using a source-available license).
@element @nemobis 5 MEUR/yr for 5 years sounds small on the EU scale. One possibility would be to convince one of the ERICs, arguing that EU-scale-FOSS-Synapse is crucial to EU research infrastructure [1]. Another would be to convince EOSC that EU-scale-FOSS-Synapse satisfies "seamless access" and #FAIR in supporting research and justifies a 25 MEUR/5 yrs budget [2].
@boud @nemobis Interesting. @matrix tried to pitch for EDIC funding about a year ago, but it went nowhere - it looks EDIC is focused on funnelling money from EU->Govts rather than to projects. Meanwhile âŹ5M/y is too big for NGI. Hadn't come across ERICs; will check. If we'd had baseline funding for Synapse dev we wouldn't be in this position.
@Tadano @troed @element @skylar I don't know the soft nor the details, so take it with big salt because details matter in these things.
AGPLv3 and GPLv3 can grant some permission thanks to section 7, like allowing binding with third party code (like a plugin) without changing its license. The whole program has network properties of AGPL, including the soft in a bigger one triggers the (A)GPL, but you can have non (A)GPL modules.
This is the same section allowing a compiler to produce non-AGPL code for ex.
Still, it means that you can't put arbitrary barriers in the AGPL code without people being angry and removing them, or remove features without people putting them back in a fork, as in any free software.
And for the company doing that, I would look at their governance model: if they are VC funded/part of Big Corp, the whole floss part is likely just public relations in any case, and that model ir a pure AGPL is not a very strong clue: there's a lot of cases where it happened in the last years, with whole code base going dark. See for example the recent case for Puppet.
Well actually, whatever the governance, a company doing that is not a free software good citizen.
@fanf42 @Tadano @troed @skylar the situation here is that Element releases its FOSS as AGPL, but retains the right to dual-lic as proprietary (in order to raise $ to fund FOSS dev) via a CLA, which has the term that any AGPL contribs must be released as FOSS (to avoid rights ratcheting). Yes, Element is VC funded, but the last 10y of releasing almost everything as FOSS may give some confidence that we arenât bad actors.
@f Fwiw, weâve literally routed tens of millions of dollars into building out our software and contributing to Matrix as FOSS over the last 11 years, and we expect to keep doing so. But getting large scale funding to work on FOSS is a nightmare, despite how much we like the ideology, and if encouraging enormous deployments to fund FOSS by selling them some proprietary value can help, pragmatism wins.
@element I get that making money off FOSS is hard, but you really look like a sellout.
This isn't a really easy topic, but I hope you guys will look at other options as much as possible.
I'm widely inexperienced in monetization and all that stuff but selling commercial products like a local database seems okay although I don't quite understand it (I'm thinking of MongoDB Enterprise stuff). As for a matrix client and a communication platform, it just seems weird. Just my opinion.
@f can see why it might look like a sellout, but using a small amount of proprietary software to try to fund FOSS dev might turn out to be a necessary evil in this instance. The HN thread at https://news.ycombinator.com/item?id=42752402 has a lot more context.
So... you just reinvented paywalling?
What the actual fuck, how is Matrix an open protocol if its maintainers paywall its features
@laxla Matrix is an open protocol because anyone can, and does, implement it - and Matrix is managed by @matrix.org, not Element. Synapse is just one implementation, and no features are being paywalled here (unless you count huge-footprint elastic scaling as a feature).
@pettter @laxla @element this is just not true, fwiw - we started publishing the spec in Oct 2014, one month after starting Matrix in Sept: https://github.com/matrix-org/matrix-spec/commit/556e3f8a7176a88bf4477bef6a96d4a9adbf5ad0
Synapse is just one implementation
Isn't it affiliated with matrix.org? That turns it official (even if it's not associated with the .org directly), with an expectation that it'll remain FOSS.
Element and Synpase are the main Matrix client and server; they're the first in Matrix's implementation list. That creates expectations of officiality.
unless you count huge-footprint elastic scaling as a feature
I do, actually. Scaling is a massively important feature for a social network meant to run government websites.
@JmbFountain @laxla Synapse isn't a reference implementation (and hasn't been for years), and there are no user-facing features here which are paywalled: just massive clustering. Meanwhile the protocol is @matrix.org and is 100% open, just as ever.
@alxlg @laxla sounds like youâre talking about state res v3, which you can read about at https://matrix.org/blog/2024/12/25/the-matrix-holiday-special-2024/#the-future. Progress has been slow due to lack of $ for maintenance (cf this whole thread).
> Isn't it [synapse] affiliated with matrix.org?
Not since 2023, when Element had to stop donating Synapse dev to matrix.org given lack of funding for it: https://element.io/blog/element-to-adopt-agplv3. These days Synapse is just a vendor impl.
> I do, actually. Scaling is a massively important feature for a social network meant to run government websites.
Well, if we had another way to encourage massive Matrix deployments to fund dev, we'd take it.
@element @JmbFountain @laxla that's not really fair to say. While synapse is "officially" not the reference implementation, it practically is. If synapse chose one way to interpret the spec, other server implementations need to follow that, or run in to incompatibilities with basically all of the matrix ecosystem. You have created a practical monopoly on server implementations with synapse without a feasible migration path to another server.
Not since 2023
Well, I haven't heard about it. Interesting.
Well, if we had another way to encourage massive Matrix deployments to fund dev, we'd take it.
Well, I see two options.
But the paywalling approach is just, why? Like, your corporate/government users have a financial incentive to pay for maintenance anyway, no company is gonna self-install with no warranty whatsoever if they can just not.
This'd be fine if you weren't still the main, semi-official, Matrix quote "vendor".
> Become non-profit and offer a 'membership' fee that'll provide branding usage rights.
The Matrix.org Foundation is such a non-profit, selling membership fees (matrix.org/support) - but that didn't generate enough $ at all to fund Synapse dev. Branding rights aren't valuable to commercial deployments, given they typically whitelabel.
> Offer consulting/installation/pay-for-feature services.
We have, but one-off gigs are not sustainable and undermine the underlying product work. "Pay for feature" is particularly tricky, because you don't necessarily get to pick the feature, and then have even more features to fund maintenance for. https://speaking.unlockopen.com/nhCZtn/1-billion-dollars-for-open-source-maintainers#sDmtL1I captures the problem well.
> Like, your corporate/government users have a financial incentive to pay for maintenance anyway, no company is gonna self-install with no warranty whatsoever if they can just not.
That's what we thought. But what happens is that a big System Integrator turns up to the govt and says "hey, we'll run your Matrix instance for you!", and thinks they can run FOSS and maintain it themselves. Zero $ goes upstream to us, and then you end up with https://www.heise.de/news/Probleme-mit-Open-Source-Videokonferenz-Tool-Hessen-fuehrt-kurzfristig-Webex-ein-10217839.html. And it keeps happening.
@element alright then. It is, however, a dangerous path you're taking. Lots of companies have been through this and lots have gone full corporate.
But, the question that begs to be asked is: how can these SIs approach the government before you do?
@laxla The SIs are often the only ones qualified to sell to the government; they may already be under contract in an existing general purpose agreement, or they meet the necessary size criteria (turnover, # of employees, insurance thresholds). Plus to successfully deliver a huge deployment typically *does* require a SI to do all the glue legwork. But given a chance, it is just too tempting for the SI to save costs by not paying the upstream. So: you have to give them something to pay for.
@laxla agreed that it is a dangerous path though. hopefully this thread shows how we're thinking about it, and trying to maximise FOSS while also being able to pay the bills.
@element You probably need to get help from an existing Matrix user who's involved in Horizon funding, otherwise this is impossibly hard. What about German healthcare?
The website https://gaia-x.eu/ got confusing, might still be worth joining.
Any project needs padding to cover the academic partners' fixed costs. Only GĂANT gets significant money for infrastructure https://cordis.europa.eu/project/id/101194278; in https://cordis.europa.eu/project/id/101147319 , https://cordis.europa.eu/project/id/101129751 the main suppliers get 2-10% of the pie.
@element It would make sense to try coordinating with @edri for looking for EU-level or EU-national-member-level financing. The case for supporting matrix software development is strong, but administrative-political coordination is needed. EDRI should have some good ideas and practical knowledge.
@nemobis @ilumium
@boud @element @edri @nemobis At EDRi we indeed advocate for *much* more EU funding for free and open source software that is part of the digital commons.
Just this week: https://edri.org/our-work/meta-and-x-are-going-rogue-here-is-what-europe-should-do-now/
We would love projects like the #Matrix Foundation to be on the receiving end of this. There is unfortunately heavy political resistance despite all the talk about #DigitalSovereignty.