Looking down

[CultLeader]

For my thoughts are not your thoughts, neither are your ways my ways, saith the LORD. For as the heavens are higher than the earth, so are my ways higher than your ways, and my thoughts than your thoughts. - Isaiah 55:8-9
And he said unto them, Ye are from beneath; I am from above: ye are of this world; I am not of this world. - John 8:23
I am Alpha and Omega, the beginning and the ending, saith the Lord, which is, and which was, and which is to come, the Almighty. - Revelation 1:8

Let's talk about two directions software is developed in this world. Say, you develop an open source library. You're in for the world of hurt. How many different people in different contexts would try to use your library? Some idiot in golang book (yes, I've read a golang book just to hear them out, that reassured me even more that golang is trash and one day I might write dedicated post roasting golang) claims that if you want to make useful software it has to be generic. There is that evil word again.

The truth of the matter is if you make stuff generic your things are inherently limited. You are looking up. Looking up means like looking up to the sky, whether it will rain or not, you don't control that. The people that use your library are the ones from the higher context, in a specific organization looking down into your library whether it is usable in this context or not. Long story short, users of your library have all context to use it so they have much easier time to decide what to do. They don't have to use your library, for specific case they could develop their own custom component.

As soon as you put yourself into context of what you want to achieve you start looking down. You have full control of your project. Well, I guess someone could say that as I'm developing Eden platform (I'm roughly 60% there I think, developing infrastructure tool that has cloud/provisioning/versioning/logging/monitoring/ci/cd/security/applications/load balancing/databases/queues and etc. integrated together is quite a lot of work, believe me) I am imagining all the context.

In Eden platform everything is assumed, private ip scheme is assumed. Routing amongst subnets is assumed. I assume datacenters will have certain ip prefixes, like 10.17.0.0/16 for one datacenter and 10.18.0.0/16 for another datacenter. Hence the routing is simple. I assume datacenter can have up to 65k hosts, all inside /24 subnets inside datacenter like 10.17.1.0/24 for one subnet and 10.17.2.0/24 for another. I assume every datacenter has at least two wireguard gateways to connect to all other datacenters which is checked at compile time and you must fix it to ship. I use OSPF in FRRouting to route across subnets and regions and I don't need anything else.

So, developing Eden platform forced me to think in global ways of how one organization can develop everything as a single unbeatable monolith which supports infrastructure of around eight million hosts. I am thinking globally. And that required some refactorings on my part early, to figure out how regions, datacenters and interconnections between datacenters play together for all infrastructure. I was forced to go through those decisions early, not when you need to expand to multiple datacenters in production and nobody has clue on how to do that and then it takes years because of poor early decisions.

The inferior alternative to looking down is to look up, be someone's bitch. For instance, I could have used something like dnscontrol for DNS and pray they support such and such provider for all dns records. Well, that's not good enough if you want to make all assumptions and have all control. So guess what Eden platform does? That's right, we run our own DNS servers, we use rock solid BIND that stood a test of time, hence we can look down. We do our own routing. We do our own builds and rollouts with rock solid and reproducible Nix build system.

I'm not yet into this at the time, but plan is also to just generate all the terraform code needed for every datacenter. Say, you'll specify in datacenter `implementation` column that aws is the implementation and then Eden platform will generate all the terraform for all the machines in their appropriate subnets which obey Eden platform ip scheme and they'll be automatically connected via wireguard to the rest of datacenters of Eden platform, which might be on premise, google cloud azure and etc. And if more than one AWS datacenter exists they'll be connected with native AWS transit gateway instead of wireguard. We can do tricks like that just because we have all the information about our intrastructure, and don't have to limit ourselves with wireguard across datacenters.

It's a lot more work, than say, some absolutely worthless project that I don't understand why anyone would use - FlywayDB. Typical Java crap that... Executes schema migrations. And a lot of people were convinced to use this nonsense. Eden platform rolls its own schema migrations because it's very simple to do BEGIN; execute all migrations; COMMIT ourselves. Database already provides these abstractions that make it easy. However, if key value store was used it would be much more complex.

So, I do not want to release half assed trash like FlywayDB which does one thing and you need to integrate it to everything else. For Eden platform to be useful it must connect and make everything to work together from day one. Eden platform should be all that you'll need, where everything is integrated and working together since day 1, just like Apple strives to do. I want to make assumptions about all the inrastructure, hence we must take control of all the components. When we take control of all the components we are looking down and our job becomes easier, while if we were looking up, such project as Eden platform would be practically impossible.

Man is taller than a woman and looks down on a woman. Say many men use the woman, like the library we talked about. How do you call such a woman that many men looked down upon and used her? We call her a harlot, and rightfully so.

In a perfect design for which we strive, if you look up you can only look up into one direction, have only one master to whom the component will fully submit and not be influenced by any other component. No woman can ever be shared between two men, it is an abomination.

Of course, Eden platform uses third party components that are generic, used up, and because they are not aware of the certain context, suboptimal in certain places. This is just the beginning to get things up and running.

Once Eden platform is built and running in production, then we can allow ourselves to have custom, better optimized components in certain context where generic ones are limiting. But that is very far into the future.

Long story short, if you want to make your life easy, do not be like an open source fag that desperately tries to support every new itch of platform under the sun. For instance, I'll likely release Eden platform only under NixOS because I consider other operating systems a waste of time. To make your life easy, start looking down, globally, about the global context of your app so you can make assumptions and you'll avoid so much trouble down the line where most other engineers will be bogged down by meaningless indecision inducing details.

Have a good day bois.