And the angel which I saw stand upon the sea and upon the earth lifted up his hand to heaven, And sware by him that liveth for ever and ever, who created heaven, and the things that therein are, and the earth, and the things that therein are, and the sea, and the things which are therein, that there should be time no longer: - Revelation 10:5-6
I'm at the point where I'm developing eden platform, which will be the first open source version of the pattern. I'll likely be able to release the early achy breaky version as a proof of concept in half a year.
I came across an interesting thing when developing this project. Now my infrastructure and apps are all data. And I picked nomad to be the scheduler, vault storage for secrets (I run full hashicorp stack and I think kubernetes is cancer). Same thing that happened with Java and C# happened with Kubernetes and Nomad. Java is a complete and utter garbage of a language, C# had the hindsight of all the mistakes Java made and used it to create paid but much more superior product. Kubernetes also is the first Java, yaml hell and Hashicorp used its bad practices to create much better and pleasant orchestrator to work with.
When I was developing the pattern of my infrastructure I had a very specific problem I needed to solve.
Nomad application needs to access a secret that depends on database existing. If you run infrastructure the first time, neither database nor application exists. We're still in the planning stage.
So, it would be very complex if first we deploy database, then generate passwords and only then put password to vault and application can access it. Nasty chain of dependencies.
Instead what I did, logical plan of all the applications is built in memory. Application states that it will want to connect to the database X. Then code is generated that secret simply exists there in the nomad .hcl job file. Then also code is generated to provision that secret, generate random 42 character string and put it in vault. Code is also generated to provision that database user to access it.
What typically happens in companies, you need access to the database. All things happen in chronological order:
1. Database is provisioned
2. User credentials are generated
3. Credentials are provided to the user
4. He uses the database
What, I did instead, everything is happening at once. Before even application is deployed it knows it will need so and so secret to access it. Before database is deployed it knows it will need so and so user for application which doesn't even exist yet. This way, with the pattern, I see everything at once. I see all the applications before they even exist. I see all the databases before they are even provisioned, and they perfectly work together the first time.
This is a true godlike power, knowing and having all information at once in one place in this small infrastructure world I am creating.
Now all these times where God predicted something will happen in the bible, why God has infinite power or how God can nudge anything to happen in his direction make perfect sense. God sees everything at once, there is no before and after in the eyes of the LORD.
But, beloved, be not ignorant of this one thing, that one day is with the Lord as a thousand years, and a thousand years as one day. - 2 Peter 3:8
LORD has the higher ways we cannot understand:
For my thoughts are not your thoughts, neither are your ways my ways, saith the LORD. For as the heavens are higher than the earth, so are my ways higher than your ways, and my thoughts than your thoughts. - Isaiah 55:8-9
God sees everything from above we can't see:
And he said unto them, Ye are from beneath; I am from above: ye are of this world; I am not of this world. - John 8:23
We, as flesh, think only of one dimensional world. What we see locally is what we react to, can see and touch. What we perceive as time is simply our restricted vision which is disconnected from the entire view of the universe that God has. One day I will also see as God sees, know as I am known of God.
For now we see through a glass, darkly; but then face to face: now I know in part; but then shall I know even as also I am known. - 1 Corinthians 13:12
So, like I mentioned, the pattern framework that I'm building will be the first of its kind. And as I mentioned, any design worth anything at all will have to inevitably resemble divine patterns found in heaven. This framework will have unlimited power within its own respective universe.
- We will know what ports are used for what service before infrastructure is even deployed. Port conflicts will be impossible, and we won't need to spin up machines to test this.
- We will know that too many services are scheduled on one node with memory requirements that go above bounds of machine.
- We will know that 100 applications will perfectly interact with binary queues with perfect typesafety before anything is ever deployed.
- We will know that queries of every application work and hit indexes before anything is deployed.
No more spinning up cluster of machines to see if things work together, all of that will become automatically generated assembly perfectly interacting with each other.
I'm very excited about the future of eden platform, and can't wait to release this project to the world. There is nothing like this.
Happy new year by the way!
