Main Menu
Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - CultLeader

#16
Development Tips / Responsibility
December 11, 2021, 11:45:47 AM
I want to talk today about the responsibility and competence spectrum permeating all of the nature below:



Okay, so, on the very left there is utter masculinity. Think from the God's perspective, absolute power, absolute wisdom. God knows everything about everything and doesn't need anyone, everything that someone gets from God is by his grace toward that someone. This is the pefect, most possibly masculine image of God.

On the utter right there is utter femininity. A beautiful, submissive woman that is led by a masculine husband from the very left of the spectrum. If we talked about God this would be submissive followers, but principles are the same. Designs on this earth are fractal and self similar to patterns in heaven. What is on earth is a smaller copies of what is in heaven.

On the middle of the spectrum, where, in ideal world, no one should be, are either masculine, self sufficient women (disgusting) or weakling, faggity, feminine men (disgusting). These examples occour in nature, for instance, certain spiders where small, weak, feminine man is worthless and is eaten by a female after sex. Needless to say, God potrays these examples of how things ought not to be done with man which is made in the image of God.

Why is that? Why such opposite extremes? Why much more is required from masculine spectrum, to lead and know, to have strength and take care of the feminine spectrum?

Like we talked about generic vs specific thesis, to make the simpliest possible solutions you have to have all assumptions. To have the most possible assumptions you have to have all the information. Only such entity, that has all the information and all assumptions can make perfectly simple design decisions and impose simplest possible interfaces for the feminine plane to use them.

That is what meta executable in the pattern achieves - it has all the high level information about the project. REST Endpoints, queries, periodic tasks, all databases, their indexes, frontend elements etc. etc. Once we have all this data in meta executable, what would otherwise be a complex flow of getting/approximating this dynamic data from something somehow (like querying a database of executed query logs to analyze if queries are hitting the indexes - we're way too late, we could already be writing post mortems by now) now becomes trivial to solve as we have all the data we need defined in the project compile time in the meta executable.

All the dynamic metrics analysis systems from our standpoint are just generic abominations, which will never do the perfect work as we spake in generic vs specific thesis. They're neither fully encompassing, masculine, godlike enough, so they could provide everyone with perfect solution without extra integration code (they could, if they were part of the masculine plane executable in the pattern), neither they are perfectly simple, where you could provide typesafe API for them as consumers of the feminine plane. They're neither here nor there, hence, cannot be in the perfectly simple and beautiful feminine plane (like a beautiful, feminine woman), neither can they belong to the perfectly rational and all knowing and encompassing, infinitely powerful masculine plane.

Consider logging systems of today, elastic search + kibana. For the application logging all that is given is text. Output a line, then parse, possibly using complex regex who knows what. These will end up as separate parsed fields in kibana. Then make a kibana dashboard for every new crap app that comes out interpreting those plaintext messages of anything. And the work is never ending and certainly not perfect. This is because today's leftist faggots software devs cannot think straight, they provide generic output from the application and generic way to analyze that data which brings in bunch of complex caveats because it is not a specific solution.

Let's think about this problem from our perfect godlike meta executable, what we could accomplish with it instead?

1. Let's define every log message by type for every app in the masculine plane
2. Let's generate CapnProto schemas for every log entry and log only very small binary data that is the essence of the log message, using very little bandwidth in Kafka.
3. No more text, only possibly description next to log entry in the meta executable. Or, in meta executable we could specify formatting message how to visualize it for humans.

   Say, log message is how many eggs for what price someone bought in a store:
 
  struct LogMessageOfBuyingEggsInAStore {
    user_id @0 :UInt32;
    number @1 :UInt32;
    price @2 :Float32;
  }


  And we could provide a template string in the meta executable, with log entry of how to format this minimal, non text data in some GUI output:
  "User {user_id} bought {number} eggs in a store for price {price}"

  We could check that all of that is logical and works in compile time, in the meta executable.

  We could check that template variables exist for this log message, that we don't use all message variables, and so on and so on.

4. We could generate frontend to perfectly display our log messages according to the message types, no need to parse text.
5. Messages logged with perfectly typesafe function from the interface user in the feminine plane:

api.log_buying_eggs_in_a_store(price: 1.23, quantity: 7)


6. We could even attach alerting if we wanted to the log entries, if some error is really bad, we'd get an alert.
7. Since messages are CapnProto we'd spend 0 CPU time parsing them when dumping all them to ClickHouse

See how much more powerful solutions we can come up with that no soy leftist faggot could ever think of just by giving more power to the masculine plane where it rightfully belongs? And simple, feminine plane would only get typesafe functions on their api to log these entries? And leftist faggots tend to have the idea that something is either efficient or elegant, here we get both, efficiency with CapnProto and elegance of defining very high level abstraction for log entries where we don't even need to write more code once we developed it and things just work by adding more log entry types!

Just like a beautiful feminine woman could feel weak, doesn't need to have much wisdom inside the arms of a masculine husband who takes care of her and the family.

Like I mentioned, today, most software developers stand in the middle. They know little a bit of masculine plane, technical details, yet they feign utter ignorance when matter comes to making all things work together. For them it is unforgivable sin that database schema, queries and transactions should live together in a single codebase where they could be perfectly analysed together! Such faggots spend decades in meetings, figuring out the most complex possible flows in a rails codebase, where queries need to be checked for correctness, but also all the logic of interacting with the database must live with rails too because muh precious ruby faggots say "it would be unproductive to develop if we don't know database schemas". As if productivity is possible in the long run when developing with dynamically typed programming language. And craziest abominations are conceived, ultimate job securities that are then implemented for multiple years with no end in sight because more and more dynamic problems keep popping up.

The alternative is the pattern, all database schemas live in meta executable, all queries and transactions live in meta executable, everything is checked before the production with possibly enabled parallelism for performance and only if everything is ok, only then typesafe user API's with implementations are generated and everyone lives happily ever after. But no, let's create unmaintainable, fragile, complex abominations instead.

Like I mentioned time and time again, paganism is a mental disorder. God mocks feminine men in nature, think of all the spiders where male is small and female is huge and eats spider after sex. Such men, such faggots, if they don't fulfill the natural responsibility of calling the shots and leading the family have no purpose as nature shows us and do deserve to get cheated on, do deserve a used up slut of a wife, do deserve to ignorantly raise not their own children conceived by a cheating wife, do deserve to be only used for their money and do deserve the divorce court. These faggots are trying to enjoy in front of their women what only feminine plane was intended to enjoy - being weak, being incompetent and women ruling over them. And woman, who tries to stick her nose into a masculine plane, being the bread earner, calling the shots, being semi-smart (just a little bit smarter than feminine woman in reality, but will never reach the levels of the greatest inventors) does deserve to get smashed and dashed, does deserve to get her heart broken by the chad train that runs through her day and night in her youth, does deserve faggity, worthless, unattractive, feminine husband at the end of her youth and does deserve bearing weak and unstable children, her daughters being raised as worthless sluts and her sons raised feminine faglet losers.

None of these two variants, feminine man or masculine woman will ever be a good solution. Most software developers today fall in the middle, in the abomination which shouldn't exist. Hence, innovations are slow. They are a little bit smarter than the general population, but from the standpoint of a masculine plane all of them are insignificant small turds who will never develop anything decent.

Just think of Nikola Tesla who invented our entire modern electrical grid. After Tesla electricity innovation has stopped, same things that were used back then when he invented them are used today. Tesla knew it all about electricity, he was the hardcore innovator and brought most of these advancements to us. No wonder his labs were raided by secret agencies. But think about what that means - today software developers are brainwashed with the idea that no, you can't be the single innovator, innovations come when lots of "smart" people put their stuff together out of pieces and everyone is supposedly needed to make breakthroughs. Reality is much less romantic, you get miles ahead of every imbecile, you become prodigy, and obvious, beautiful, simple, much more advanced solutions just keep coming to you because everyone around you is a braindead and brainwashed monkey damaged by a few years of academia.

When I was in a university, there came a subject about God object that knows everything and it was told that people shouldn't do that. You're instantly told you should not strive to be similar to God. Would God be pleased if people, who he claims were created in his own image, would try to be similar to God? Of course he would be pleased if every man tried to fulfill his image. Yet, for monkeys in university it is told to do just the opposite! What are the odds! The pattern, which could easily solve most complex problems in the most concise and elegant way, given that you represent all your problems as single database of data (the dreaded God object), where you could easily analyse and check everything for consistency is discouraged from being used so software devs would keep being miserably stupid with 99 programming languages they have to learn and drowning in problems of diverse dynamic garbage interoperating with each other.

That being said, make sure masculine plane figures everything out, so there would be utter simplicity and beauty in the feminine plane, just as it is a natural and the perfect divine way to do this as is evident in all of creation.

Good day bois!
#17
Today I want to talk about another issue leftist software developers today mess up. The wording "Proudly found elsewhere", "Humble thyself and reuse", "Trust libraries written by experts" etc.

I want to explain why this approach becomes meaningless the more mature and old your project is.

You see, when you want to start a project you are attacked by countless choices for a web framework, databases, queues, infra management tools and what not. Examples seem sweet, "Look at this common case, so little code, so shiny! Use me, this framework and be productive and happy!". Every framework offers its own abstractions which it deals with and shows these as beautiful.

Devil is in the details. Now ask yourself, are developers making their frameworks make them to you? No, they make it for a group of people. And that group of people can vary wildly in what they use. Hence, any third party framework can only be generic to the lowest common denominator which satisfies a subset of people. We discussed in specific vs generic thesis how generic things will never be perfectly simple since they are not specific enough for the problem at hand. You can cut salami, wood and bang nails with an axe, but axe would be a generic tool. What you want specifically for the most convenient job is a knife for salami, axe for wood and hammer for nails.

So, unsuspecting developers get lured into yet another web framework with all its subtelties and another baggage of knowledge needs to be established in the brain until you become productive. To make matters worse, if the company becomes successful, it becomes painstainkingly obvious that the framework was never intended for multimillion lines of code, like ruby on rails. Having to maintain millions of lines of code written with dynamically typed language would cause me to become suicidal. But it just becomes too late, and people start huge projects, to split their dear monolith, no refactorings are ever guaranteed to succeed in ruby and the little framework that started so shiny with "look ma, so little code" becomes unbearable, bug ridden burden to the entire company, simply because, people who started with the framework didn't know any better, had no hindsight, and the entire company has to suffer because of early poor technology choices.

What is the alternative you ask? Glad you asked! The alternative, is going to the complete opposite spectrum, which is the pattern. Owning most of your abstractions, like infra management tools, your web framework, your frontend reactive framework. And that sounds crazy at first glance, to someone who doesn't know the power of the pattern and how it makes all of your abstractions much simpler then you'll get anywhere else just because they solve more specific problem to your needs.

The only reason I'd ever pick a generic framework like rails is for short term projects, or prototyping, if you want to get up to speed to see if your project can be successful to begin with. Then, as soon as I know that we're dealing in the long term game I'd start working on the pattern solution for the company ASAP.

You see, it is not obvious to start with the pattern, to own your abstractions. But once you get there, and you have a successful company with the pattern, and you deliver stuff so fast and correctly that everyone's head is spinning and you solve such mountains of problems which little strokes of code that work with your well defined data - no generic framework will ever be able to compete with that.

Which generic framework will offer you at a stroke, in few lines of code to, say:
1. declare a new application
2. declare which typesafe queues it will use
3. declare which typesafe database interactions it will perform
4. have it instantly all monitored according to the abstractions the app uses
5. deploy it in production and it interacts with the rest of your ecosystem correctly the first time

The answer is none, all needs to be done by hand if you take up some used up third party bycicle. And it is error prone and unproductive. The pattern can do all that, but your typesafe rock solid abstractions have to be built, which takes some time for your specific solution, but the benefits will be enjoyed forevermore. You can even have your team names in the pattern and assign enum on which team is responsible for what service. The sky is the limit of what can be done with the pattern.

And every reasonably sized company ought to have their own meta executable with the pattern with their specific use cases. For instance, the pattern of a 3d game developer will be radically different from the pattern of web application shop company. But in their own right, since their meta executable is specific to them, you are then free to not bend to the wills any framework tries to impose on you, but what will be best for the company.

Imagine checking just in one executable that entire network that spans any number of datacenters that no switches have loops, that there are no duplicate ips, that there are no duplicate ports on any server you have, that on any given server you do not exceed declared RAM capacity for what applications will be running. Imagine enforcing via iptables rules that service a can only talk to service b only with typesafe interfaces. All ran in few milliseconds, without spinning up servers in the cloud, just for the fact that you have your problems defined as data and you can reap all the benefits of having complete and total control and knowledge about your entire infrastructure. Don't like some abstraction? Delete and refactor your entire infrastructure at once and be done! No need to synchronize and think about 5 different endpoints that will not work once you shutdown the service.

This is unbeatable. People narrow down their projects like nutjobs, they create service to control dns but it doesn't work with anything else. Then they create more crappy, narrow scope services for orchestration, for network proxy, for CI and nothing ever works together and everyone needs to patch that garbage day and night and praying everything will work.

Why not have all that data, about your services, about your network, infrastructure, databases, their tables and their queries in one place where meta executable could ensure everything is working perfectly together? Leftist degenerate mindsets of today will never figure it out on their own.

So, for any longer term project, you should build the pattern with specific abstractions to you and have perfectly bonded infrastructure to your company. The initial cost will be greater than starting with out of the box framework, but once you get through a steep start of building abstractions you'll have absolute control over your project and abstractions that blow out any third party garbage out of the water. The start with the pattern is slower but the rewards are infinite.

I'll just illustrate what productivity looks with the pattern as the project ages and matures versus taking some garbage off the shelf which was never customly tailored for your company specifically



Have a good day, bois.
#18
My Rants / MOVED: What actually matters
October 24, 2021, 04:08:10 PM
#19
My Rants / MOVED: Dynamism is Cancer
October 24, 2021, 04:08:01 PM
#20
Natural Philosophy / MOVED: Captain Hindsight
October 24, 2021, 04:07:34 PM
#21
Development Tips / Captain Hindsight
September 16, 2021, 03:23:32 PM
Today I want to talk the most foolproof and rock solid way to predict the future - the hindsight.

It may sound obvious, but if something is obvious, has obvious advantages and yet not many people use it and believe the benefits of it, maybe it is not that obvious?

Today a lot of software developers start building complex abstractions, especially in OOP, without having a clue what practical needs actually are. I'm talking mostly about Java projects, which abuse tons of interfaces like I mentioned in specific vs generic thesis.

The problem is not wanting to abstract something to high levels. The wish itself is amazing and is core principle of the pattern which solves tons of problems that cannot be solved by 99% software engineers today.

The problem is that people miss the abstractions, they build them without hindsight of do's and dont's to arrive at the simplest possible solution to the problem. What is inevitable, that by spending time on board and designing something that looks good in imagination yet is quite a flop in reality, because they don't have hindsight.

How to build the abstractions perfectly by acquiring the hindsight?

Do not build the abstractions right from the start. Before I had the pattern which allows me to solve any programming problem in the most concise elegant and most efficient way I didn't have abstractions.

Say, today, I define typesafe rest endpoints and frontend code gets generated and I implement typesafe functions in the backend. How did I think about that? In hindsight, it is obvious. But to arrive at this conclusion, as a poor soul that I was, I simply wrote OCaml code for the backend, and I separately wrote OCaml code to the frontend. I parsed url arguments by hand, was doing mistakes and I just thought to myself - this is bothersome and error prone work. Why can't we generate code that calls the frontend, parses typesafe responses and you don't deal with json at all?

So I did, I did not have this powerful abstraction at the start. I was doing that work by hand, then there was actual, practical need for the abstraction and then I built it and enjoyed the benefits ever since.

Same happened to me when interacting with postgres. Yes, OCaml is amazing, typesafe, as long as you work in its typesafe plane. But, if you use database, you send a string which is not typechecked to the database, hence, you must run the code that uses database to know it will work. So, I had OCaml, that is wonderful language, yet I am suffering from pitfalls of dynamically typed languages I cannot stand by using plain SQL in OCaml code. I didn't want to write tons of tests either like ruby nutjobs do.

Again, from this practical need, I decided to solve this problem with the pattern, every interaction with database via SQL queries must be defined as data and ran against test data with test arguments to ensure query succeeds and returns results. Not only that, I went step further, I didn't restrict functions that were to be generated to just return bunch of strings you have to parse - I made sure all the results from generated code were parsed into typesafe OCaml types.

So then, I brought database into OCaml world, I write SQL queries that are checked in meta executable for correctness and generate typesafe functions that accept typesafe labeled arguments and also return typesafe results of array of structs. From OCaml standpoint, I completely eliminated risks of using raw SQL strings in OCaml and I am in wonderful typesafe OCaml land again.

The need for both of these abstractions arose from hindsight. Not to end up like Java projects, I NEVER BUILD ABSTRACTIONS BEFORE I KNOW I PRACTICALLY NEED THEM. I do the work by hand in the start, once I see I copy paste a lot of stuff that is mostly repeating, at that point I know what I can abstract and what code I can generate and hence I can never make mistake of producing useless unneeded abstractions.

Or, think of Java vs C# - Java was the first one to make all the mistakes and C# did everything better. Or think of C++ vs Rust - Rust had tons of hindsight about bad practices of C++ and eliminated them.

It doesn't matter whose hindsight you use, actually, to use someone elses mistakes to your advantage is cheaper. But, with things like the pattern, as I am the only person I know who does this, and I cannot understand why everyone doesn't do this, I have no hindsight of others to work on. Hence, in such cases, I have to make mistakes myself and then, with hindsight, turn those into abstractions that actually work.

I do not pretend to know everything about good practices with the pattern, I am the first person I know to use it, hence, the area of the pattern is an utterly uncharter experimental territory that needs lots of exploration to be made, and why I created this forum, to help find more fellow pioneers. Am I scared? Not really. You see, everyone else around me is riding a horse and I am the first person to have the very first car. Will it have its quirks? Sure, but I'm already ahead of everyone else that has a horse.

So, long story short, if you want to make practical abstractions, that will be joy to use and will have no risk of becoming overcomplicated, overengineered or useless - always base your abstractions on hindsight, be it someone elses or yours. Don't be an idiot who reads whitepapers all day written by people without practical sense thinking you'll do yourself a favour if you implement something purely theoretical and not based on practical hindsight.

Peace bois.
#22
Development Tips / What actually matters
August 14, 2021, 08:57:10 AM
Today I will talk about what actually matters in software development to understand why we do what we do.

If you want to choose a coding style there are many options available, say Object oriented programming, domain driven development, test driven development, liskov's bullshit principle, choose convention over configuration and so on and so on. How to know what to use? Today, I want to simplify people to think about just one thing basically to get max correctness and solidity from their projects.

And this is the following spectrum below:



Basically, all that actually matters is how many mistakes you can eliminate before your app runs in production. I'm sure most devs would like to find out about issues early not to be sweating in production fixing issues and writing post mortems. It's that simple.

So, all of our decisions in software development, if we want to strive to make correct software, we should want to know that as many things as possible will work before app is running in the wild.

We assume that every bad thing that could happen will happen unless we enforce otherwise. With this mindset, let's evaluate some mistakes that happen during application development and how can we align these decisions with this simple curve.

Using undefined variable (like from typos) in code

Nasty error. Happens so often in dynamically typed languages it is not even funny. Dynamic languages proponents say that this is remedied with tests. But tests are written by humans. Humans make mistakes. Humans may not test all paths of code. And if they do that's a lot of extra work that needs to be done.

How to deal with this problem? Use statically typed language and you will never see this, its that simple. With this solution we slid the divide between compile time and runtime a little to the right (where we would want to).

Incorrectly written SQL queries in code

Can happen in statically typed languages. Either write tests to check every flow that uses the database, or, as I do with the pattern, have all SQL queries defined as data in meta executable, test them in meta executable against test data, and generate typesafe functions to use those queries. Again, we slid the dividing line to the right.

Incorrect REST call from frontend to backend

People use naked json's with ramls, swaggers or dynamic garbage like GraphQL. This will only get you so far, this disallows for interface changes, which encourages legacy interfaces to be maintained. How to never worry about this in production?

Again, what I do with the pattern is have my rest endpoints defined as data in meta executable and generate typesafe code for the frontend to access the REST endpoint and for the backend and I can never make a mistake. Easily slid the dividing line again.

Incorrect html formation for frontend components, like buttons etc.

Dealing with lots of HTML is hard. Doing that with plain Javascript is horrible. You have no idea if things will work just changing plain javascript, you have to test it.

What I do with the pattern, is define an enum in meta executable of a certain button

Meta executable:

let frontend_resource_types : enum_type list = [
  mk_enum_type
    ~enum_prefix:"Button"
    ~type_name:"user_buttons"
    [
      "Logout", FtUserLogin;
      "ChangePasswordShowDialog", FtUserSettings;
      "ChangePasswordComplete", FtUserSettings;
    ];

    ...
]


Generated code:

type user_buttons =
  | ButtonLogout
  | ButtonChangePasswordShowDialog
  | ButtonChangePasswordComplete

let all_user_buttons = [
  ButtonLogout;
  ButtonChangePasswordShowDialog;
  ButtonChangePasswordComplete;
]


Then I have functions that have basic labels on buttons and html generation


let user_button_to_name =
  function
  | ButtonLogout -> "Logout"
  | ButtonChangePasswordShowDialog -> "Change password"
  | ButtonChangePasswordComplete -> "Password change complete"

...

let user_button_to_html
    ?(enabled = true)
    ?(data_id = "")
    ?(custom_callback = _blank_custom_callback)
    (btn: user_buttons) =
    ...
  let default_template () =
    Printf.sprintf {|
      <a %s data-class="%s" data-tag="%s" onclick="callbacks.onButtonClick(this, '%s', '%s')"
        class="waves-effect waves-light btn %s">%s</a>
    |} maybe_id btn_id data_id btn_id data_id disabled_class name
  in
  match btn with
  | ButtonLikeMessage ->
    Printf.sprintf {|
      <a data-class="%s" data-tag="%s" href="#!" onclick="callbacks.onButtonClick(this, '%s', '%s')" class="%s">%s</a>
    |} btn_id data_id btn_id data_id disabled_class name


Now every time I add button in meta executable I have to go pick a label, I copy paste html from other button example, style html any way I want and I also have a place where I have to create implementation of the button after I add it:


let on_button_click (dom_elem: Js_of_ocaml.Dom_html.element Js.t) the_btn the_id = (
    ...
    match the_btn with
    | ButtonLogout -> (
        clear_session_cookies_and_refresh ()
      )


Then anywhere I need that button I just generate it as enum and can never make a mistake:


str_gen_basic (fun append_string ->
    user_button_to_html ButtonLogout |> append_string
  )


And it always works the first time.

Basically, for one abstraction, like button, I dig a tunnel, and I have typesafe code I have to implement for it to work. The sample of buttons I used is smaller than actual project, I use tons of them. And every time I add a new button everything, its html, its labeling, its actions work the first time. I did same with dropdowns, same with restful loading components that render loading animation until server gets queried, user inputs etc. etc. I don't deal with raw html when interacting with buttons, I deal with typesafe enums.

Again, we moved the slider to the right.

Ensure kafka topic is used by someone

In 99.9% software projects today nobody is checking that and everyone hopes and prays that everything works.

What you can do in meta executable, if you have all schemas and topics defined as data, and you generate typesafe interfaces to be implemented, where you allow everyone to interact with the topics if that's the only way topics are used in your project, then it is trivial to test in your meta executable, with few loops through your data, that topic is used nowhere and you can mark queue deleted in data, and it would be executed in your infra.

It's that simple.


Conclusion

So, basically, when any bullshiter comes to you and says "we need to implement so and so bullshit pattern because so and so PhD idiot who never worked on practical problems in his lifetime wrote a whitepaper"

Show them this spectrum and ask the simplest questions:
1. How many possible mistakes this pattern introduces into mistake spectrum
2. Does this pattern move overall slider to the compile time side (like the pattern always does) or does this pattern move it to the left, where there will appear more errors in production?

Very simple, clear, easy to understand, common sense, no need to write whitepapers about it.

Have a nice day folks!
#23
Development Tips / Dynamism is Cancer
August 01, 2021, 04:23:19 PM
Today I want to talk about one of the most rottenous cancers found in codebases today - dynamism. Usually, whenever someone wants to introduce dynamism in the system, things become more complex, harder to reason about and inevitably fail in production like a crooked building it is.

Theoretical reason behind this, is that dynamism introduces lack of information in the system. If things are static, i.e. known beforehand, we can test possibilities and anticipate outcomes. Dynamism, on the other hand, by definition, introduces ongoing change in the system. Which, usually, cannot be tested and reasoned about. And flows involving dynamic mutating data are very complex, and needlessly so.

Let's take apart a popular example - kafka schema services. Say, there are many topics in the kafka and each topic has different type. How do we ensure all elements are of appropriate schema and we don't get surprises? The dynamism cancer answer to that would be confluent schema registry https://github.com/confluentinc/schema-registry . I.e. a service, that has all your schemas, and you post to that and hooray, muh schema is online.

Okay, how do we check from repository, where our code is, that we interact with the queues correctly? We have to query the production schema registry. Which, needless to say, is horrible, because it can be down, and our builds become undeterministic and fail. And, of course, another issue, is that schema registry is very limited only to schemas it supports, avro, protobuf and json. Another cancer, since this is a generic service, it supports JSON for changing its data, which is a generic and error prone data format. Could we, for instance, make schema registry check that a kafka queue can be represented in clickhouse via table? Not really, we would need to do this by hand with complex dynamic flow querying the registry.

Someone might say, hey, this isn't that bad. Well, such people also say, it is normal to spend years on data stream projects because, oy vey, they are so complex. No, they're not. Imbeciles doing them make them complex.

What is the static answer to this? DEFINE YOUR SCHEMAS AS DATA IN YOUR CODEBASE IN YOUR META EXECUTABLE USING THE PATTERN.

Now, let's go over these problems if we simply define our schemas as data inside the same masculine plane executable and what that allows us to do:
1. Builds and tests are deterministic - we have the schemas in our codebase and can check them any way we want
2. We can write code, that generates code for absolutely correct typesafe interaction with the queues, where we don't parse anything at all or are aware of the underlying data formats
3. We can ensure logical/backwards compatible schema versioning with the queues
4. We can easily check if queue is appropriate to be reflected in clickhouse (checks like if columns are not too exotic)
5. We can don't deal with JSON error prone plaintext - our problem is represented as data in a typesafe OCaml and we avoid countless issues
6. We can easily generate SDKs for mobile clients or typescript clients to intereact with our queues via generated REST services easily and without mistakes
7. We have no complex flows of testing correctness, involving multiple production services which could be down, everything is tested locally, mostly in memory, hence, less need for monitoring of production failures

Actually, the benefits of knowing all schemas beforehand in the pattern are countless. We can solve all these problems with just a few loops of checking our data for any correctness degree we could imagine. There is nothing hid. Imagine, if you need to walk in absolute darkness, where you don't know what trees are around you during compile time (dynamism) versus when sun is shining and you know everything beforehand in compile time. This opens completely unexplored space of countless possibilities of what you can do with this.

Why are we storing and cherishing customer data in the companies and value it so much? Because it gives great insights about the customers which we can use to better our business. Why we treat our code metadata, our schemas, our types like garbage? Why do we divide our precious data so far away, that we cannot use it together as a whole to draw conclusions about our codebase itself? Why is it shoved into some third party component that we cannot control and that we cannot efficiently query to check if it aligns with our codebase? Or why do we monitor such component when it is too late and something is broken and people are sweating in production because we could not check if beforehand?

All such crapware as confluent schema registry shouldn't exist. Same with kubernetes yamls, which ought to be services declared in our meta executable, where we could enforce they way they interact with the system with absolute typesafety. Same with chef and its server which contains all the error prone, non typesafe json roles, where you can put anything you want in that json and nobody checks that any recipe actually read those attributes - you only find out in production. Dynamism is cancer, it breaks the system apart, divides data from data and cripples us so that we can no longer use our metadata in incredibly powerful ways like the pattern allows to do. All configuration ought to be defined in typesafe programming language (I prefer OCaml) where we cannot represent illegal states or illegal states are captured when we test the data in RAM for consistency with the system.

So, moral of the story today:
1. Run from JSONs
2. Run from YAMLs
3. Run from dynamic configurations whenever possible
4. Dynamism is cancer and breeds insane complexities to the simplest problems

Have a good one bois.
#24
Natural Philosophy / Divine systems
July 23, 2021, 03:43:36 PM
Today I want to talk about what is a discerning factor between God's creation and between all the evildoers in this world who lie, steal and murder for their own sakes.

Consider a simple transaction, I want to buy a loaf of bread and person selling this to me wants my money in return. Transaction is completed, I eat the bread I wanted and I'm happy, the person is happy with getting the money from me. There is no strife, everything is simple. The core principle of a divine system is that every party gets what they want and they are happy.

Consider another scenario, where I have a bread factory network, that stretches throughout entire world. I'm making fixed income from everybody in the world that buy my bread. I decide that money I get, even though it is very much, is not enough for me. I buy ads, I buy scientists studies, I put tons of money into education system so I would "teach" everyone, that they have to eat three loaves of bread every single day or they will die. And that all other food is evil and unhealthy. I have deep pockets to do that, why not?

And after 50 years an entire new generation of people is raised up that only know that they have to eat three loaves of bread everyday or they will die. But wait a second, wouldn't some people figure it out eventually and call out all this nonsense I've ordered to teach these years? Sure there will always be some conspiracy theorists. But I make sure to abolish free thought, discourage critical thinking and questioning of the curriculum. I'd reward people who praise the health benefits of eating three loaves of bread in a day, and I'd punish and ostracize everyone who questions the three loaves of bread in a day theory.

But hey, both groups of people appear to be happy - I get my money and people, even though they became very fat, live on eating three loaves of bread in a day. Except, the only way I can hold on to this position, is to keep lying to everyone.

So, you probably guessed it - system I made with three loaves of bread in a day is a devilish, evil system. What is the discerning factory?

In a divine system, if you know all the facts, you choose to participate in the system of your own free will for your own benefit.

Yes, you still participate for your benefit, and you get what you want. And in you getting what you want other parties get what they want. You can be completely selfish in a divine system, doing the right way, while not trampling anyone's rights under your own feet. This is how bodies and communities form naturally from self-willing people, working for their own benefit and secondarily for the benefits of others.

Consider a husband and wife. If we know about fractality, that marriage is an image of God's relationship with his people, we know that God does everything for his own pleasure and his own glory.

Thou art worthy, O Lord, to receive glory and honor and power: for thou hast created all things, and for thy pleasure they are and were created. - Revelation 4:11

Women, before there came about many bureaucratic needless work opportunities for them, sought men who would take care of them. Of course, there are hormones and sex, but in this transaction, a man gets what he wants - pleasure from her freewill of choosing him, especially if she's a virgin and she trusts you alone with her nakedness. Also, man gets an offspring. The woman also gets what she needs - being taken care of by her man and she doesn't need to go to work and can raise children in the grace of her man. Her life becomes very simple if all the thinking is done by her man.

And, of course, if the woman wants to learn everything her man knows, she would just come up to the same conclusion, that my man takes care of me for his glory and for his name's sake and keeps me around because I bring him glory and pleasure. My husband is competent and I like him taking care of me and taking all the complexities of life out of my head.

Now, consider a marriage of an average beta soy boy to a used up slut. Does a slut bring glory to a man? So many men had her, and you were the one to put a ring on her finger? People laugh behind your back for marrying a whore? So, such marriage is evil by itself, because the man doesn't get all he is supposed to get from such contract. And, of course, sex is worse for the slut after being married to a soy boy, because beta boy will never satisfy her the way all the chads run through her in her youth. And often, such women don't even sleep with their husbands. And the woman gets money from such beta boy. This is one sided transaction, and hence, not a divine system, and hence, evil.

Or, consider the sluts that give up their nakedness to very high value men who will never commit to them. These men get sex, woman just degrades her value, develop all kinds of mental disorders that sluts develop, and don't get a ring on her finger. This is also one sided transaction, where man gets everything, and hence not a divine system. Would woman sleep with a man if she knew that his plans for her were pump and dump and she is not taken seriously? A lot of them wouldn't.

So, to create divine systems is quite simple, and decentralized #defi movement is doing it.
1. Be completely transparent about your plans
2. Reward all the participants of the system with what they want

A great example of this is crypto exchanges that used to make the bank on the transaction volume due to trading fee. Now, with decentralized exchanges, you, as an average joe, can provide liquidity to the market and receive that fee. Trader gets what he wants - he trades some coin from A to B, you get what you want - fee for providing liquidity for him to make this trade. Win, win - a divine system.

A great example of an evil system is modern medicine, well, I wouldn't call it medicine, I'd call it butchering. Read a book "Goodbye germ theory: ending a century of medical fraud" by William Trebing, where he exposes vaccines for the fraud it is which only causes you to be unhealthy and more sick than had you done nothing at all. And entire world is brainwashed, taught since high school that vaccines are supposedly heroes that eradicated many deadly diseases. Ignorant fools shame and mock you for being an antivaxxer. Yet whenever you investigate an issue you find out time and time again that entire medical establishment is a worthless industry that provides no benefit to society and should not exist. Imagine all the money they have to spend yearly to keep you brainwashed - definitely a devilish system.

Divine systems are much simpler, transparent, coherent, correct, have no lies or inconsistencies in them. This is the way of the LORD, so that inevitably, after seeking him we would come to him and give him glory for his infinite wisdom.
#25
General Discussion / Hello newcomers
July 18, 2021, 07:29:59 AM
One person registered, and said he would love to see where this will go. I appreciate that.

Also, verification system is tough, so, if you're a newbie I'll just share the answers:
Rust solved all the problems of C++ and, Java, being a complete cancer uses too much memory and startup times are horrible.

I encourage people to ask questions on the forum, publicly, as I only log in to this forum and see the messages when I publish the articles. But I can see your questions if they are public without being logged in. I'll answer everything.
#26
My Rants / High IQ is overrated
July 04, 2021, 12:18:09 PM
Today a lot of dumb people boast my IQ is so and so. They even make the fallacy of saying, since my IQ is so and so hence everything I say must be right. Good thing reality and correctness doesn't care about IQ and is not bothered with meaningless arguments from authority. Reality and real wisdom is easy to be understood, easy to be recognized. It doesn't need to be forced, people come to the conclusions of wisdom of their own free will for the benefit of themselves first and for the benefits of others. When you hear wisdom in reality you don't need PhD's to know when something is said is true, it just becomes obvious. With wisdom what you usually do is connect dots in the people's heads that they already have, sistemize their thought process and make it simpler. You do this until all complexity is eliminated and only the obvious parts that have to be there are left. That is wisdom.

Then came the officers to the chief priests and Pharisees; and they said unto them, Why have ye not brought him? The officers answered, Never man spake like this man. - John 7:45-46

Ordinary people without high education heard Jesus speak and he was bursting wisdom out of every word. They didn't need PhD's nor bible colleges to grasp what he said for the most part. Same thing said in the bible in other words in James:

But the wisdom that is from above is first pure, then peaceable, gentle, and easy to be intreated, full of mercy and good fruits, without partiality, and without hypocrisy. - James 3:17

Now let's talk about some of the dumbest, most pretentious intellectual midgets who somehow have audacity to perceive themselves as smart. That's right, I'm talking about good old leftist cucklets soy dripping software developers.

You see, a humble person that knows he's not that sharp is not a problem. You can reason with such people easily. But when an idiot perceives himself as being smart then it is much harder.

What am I talking about? For instance, in a daily life, someone made a PR where they used mutex to block access from two different threads. I left a comment "well, if you just order the code in such a way you don't need locks at all". Idiot merged anyway. And, of course, that person is perceived as smart in the team, had PhD in who knows what, I don't care. Yet the person, when presented with two options, one being use a mutex to synchronize access and another is just reorder code and you don't need locks at all - this person picked a more complex way to do things. Imagine that, a person has PhD and yet the same person is an imbecile.

And such people love overusing complex software development strategies, patterns, overengineer crap out of everything and yet these same people mock someone who works in a construction site or drives a truck as being dumb because they cannot code. Why are software developer salaries so high anyway and why are there so few of them? There is a big barrier of entry and indeed you need lots of smarts in the current climate to understand all the crazy complex mess that we ourselves have created in the first place. Imagine all the rotting Javascript in the world which soy dripping faggots debug day and night and will move heaven and earth to keep pushing abomination of NodeJS instead of some rock solid statically typed language. You see, the work of a construction worker is reduced to such simplicity with great wisdom so that very many people could join in up to speed. How long construction has been going on, for thousands of years? This timeframe has allowed for the craft of building to have all sorts of tools that will get the job done.

How looked the first house ever built? Probably like crap. And that person who built it (Adam?) likely made lots of mistakes. But that's fine, the more time passes the more practice and everything becomes easier. And of course, in the market of first houses there were very few skilled craftsmen who knew do's and dont's and probably charged a hefty premium to build someone a house, like software developers today. Eventually this workforce became cheaper.

Consider software development. How long it has been? 50 years at the time of this writing? We're in the very infancy of this craft, where first attempts bred insane complexity (by imbeciles who think they are smart) where there are very few people who could maintain it and understand it. And yet they brag about how all the garbage built is so complex that only few priests can maintain it, have much higher salaries than the rest of the people and then mock other people, who work in art that has been developed thousands of years, like construction, where everything was reduced to such simplicity and all do's and dont's are known and now understood. Some people work on software projects so long that they make a bets seeing an office across the street being built, that they will not finish this project until office gets built. And they're often right. These imbeciles cannot type with their keyboard in a wise way and make wise decisions to such extent that building with a cheaper workforce appears in front of their eyes. If that does not speak of the volumes of extreme stupidity of most software developers today I don't know what does.

Today, software development goes like this, if we used a building analogy: two people build separate walls without seeing each others work. Walls are of different heights, yet they found out about that only when trying to attach roof and it doesn't fit. This is finding out about issues in production. What is the remedy to this in house building? There is a model, a project, in which architects figure out in imaginary plane of how the house looks and model it. You can't just do all this in your head, like software architects do with diagrams and pray they work. A building, say, AutoCAD model checks if walls are of appropriate height and such problems are detected in AutoCAD so they wouldn't appear in production. And there is no question about what height wall should be, it is decided in the model. So when I look at mistakes of postmortems, like, say, database field is removed and it was used in frontend and now website doesn't work because of null pointer exception - this is pathetic. These people have no plan, no model where everything is worked out.

Such issues are worked out with the pattern, where one executable has everything related to it. We can check that database query uses indexes so it would never use sequential scans in production. We can check that query is never used if we have it as data in our plan. How fewer post mortems you would have if you checked such details before deployment and would enforce that such crooked buildings wouldn't bee deployed? 90%+ complexity I just remove with the pattern, by checking issues early and don't deal with them at all. The pattern allows you to build code like a building, you create foundations, then build more and more things on top and the code scales like a building. If there are cross context inconsistencies you know instantly instead of finding out about them in production. With the pattern you, as one developer, can do job of 100 developers and know everything will work together, hence, drastically reducing cost of the projects. Pattern is like having a building plan - todays imbecile developers build buildings without a plan! And Jira is not a plan, mkay?

And guess what? If you have well established meta executable it becomes trivial to develop with it. You need REST endpoint with the pattern? Add another row in the table of rest endpoints, define its signature, code gets generated to fill in the blank for the trivial typesafe implementation and you're done. And working in a feminine plane only you'll have much simpler job and hence, more people can do it, hence workforce for software development projects can be cheaper. And of course, leftist cuck wet dream or nightmare, more women can finally code in less understanding requiring feminine plane and get paid lower salary than few men who work in masculine plane which requires all the understanding of how everything works together, LMAO.

So this is why I don't view having a high IQ a very important thing. Once you start doing things with the pattern your life becomes so simple and trivial that you don't need to read whitepapers on how most hardcore locking primitives work. You just build abstractions that don't require them, because you see big picture earlier. That's why most leftist faglets, you can keep complaining about how women are attracted to masculine men and how nobody appreaciates your imaginary smarts in the complex abomination of a reality that you yourselves have created. And such leftist cucks will also not reproduce sufficiently in the long term and disappear from this earth themselves in due time. Complexity is not how nature works, it is simple, neither are we traveling around the sun (if you want a quick summary check this out https://www.youtube.com/watch?v=WffliCP2dU0 ), this is other thing pagan leftist sun worshipper cucks created in their mind to pervert and pollute mind about the complexities of nature which is designed simply and beautifully by the Master Builder of the bible.

Peace out bois.
#27
Natural Philosophy / The Image of God
June 20, 2021, 08:36:49 AM
Today I will talk about masculinity and how it is necessarily related to divinity. After all, I've discussed in masculine and feminine planes thesis how the concept of sexes can explode individual developer's productivity at the right place and at the right time.

First of all, let's start that God created man in the image of God:

So God created man in his own image, in the image of God created he him; male and female created he them. - Genesis 1:27

And it says that man was created in the image of God, but not a woman. Okay, how to leverage this? Think of having a son and you really like software development. Wouldn't you be very proud of your son if you taught him to code like you do and he also became a rockstar developer? That would be a very good feeling for you. So, you'd appreciate if someone followed you in your footsteps, you'd be happy that someone resembles the image of you in their young age. You'd want to reward such son of following your footsteps. You could answer all his questions about coding. But, if your son deviates from you, and becomes someone like mechanic or a street cleaner, that would be fine but wouldn't be as ideal. You couldn't help them much in their specialty if it deviates from you.

So, in the same way there exist men that are all over the spectrum in resembling the image of God. At the very beginning of the spectrum you get feminine, weakling faggitty sissies. At the end of the spectrum you have someone like king Nebuchadnezzar, who brought the entire world to his knees.

The tree that thou sawest, which grew, and was strong, whose height reached unto the heaven, and the sight thereof to all the earth; Whose leaves were fair, and the fruit thereof much, and in it was meat for all; under which the beasts of the field dwelt, and upon whose branches the fowls of the heaven had their habitation: It is thou, O king, that art grown and become strong: for thy greatness is grown, and reacheth unto heaven, and thy dominion to the end of the earth. - Daniel 4:20-22

Just like God is the ruler of the heaven and earth, needless to say, man will always be lower than God in control but a man can still be a ruler within his limited scope.

Divine attributes

So, man's attributes determine how close he is to the image of God. Let's see some of the divine attributes of the Lord:

And the LORD brought us forth out of Egypt with a mighty hand, and with an outstretched arm, and with great terribleness, and with signs, and with wonders: - Deuteronomy 26:8

1. God is infinitely strong and mighty

O LORD, how manifold are thy works! in wisdom hast thou made them all: the earth is full of thy riches. - Psalms 104:24
To him that by wisdom made the heavens: for his mercy endureth for ever. - Psalms 136:5

2. God is infinitely wise

One thing have I desired of the LORD, that will I seek after; that I may dwell in the house of the LORD all the days of my life, to behold the beauty of the LORD, and to enquire in his temple. - Psalms 27:4

3. God is infinitely beautiful

And the LORD passed by before him, and proclaimed, The LORD, The LORD God, merciful and gracious, longsuffering, and abundant in goodness and truth, - Exodus 34:6

4. God is merciful

And I saw heaven opened, and behold a white horse; and he that sat upon him was called Faithful and True, and in righteousness he doth judge and make war. - Revelation 19:11

5. God is just and can make war

But the hour cometh, and now is, when the true worshippers shall worship the Father in spirit and in truth: for the Father seeketh such to worship him. - John 4:23

6. LORD seeks people that will worship him

And he hath on his vesture and on his thigh a name written, KING OF KINGS, AND LORD OF LORDS. - Revelation 19:16

7. LORD is the ultimate ruler



As a man if you implement God's attributes you will be rewarded with love of women and offspring of a traditional family.

For instance, if you are low in the looks department, you will never fully resemble the perfect image of God, but you can compensate by getting jacked in the gym. Also, women drool over intelligent men and praise them for that. Woman will choose to be with a man that is worthy of worship and will most closely resemble the image of God. That's why they try to get the ultimate best mate that they can, and men only look for a few things in woman. Sure, a bad boy may have only trait of being beautiful and ruling to a woman, and woman will naturally fall for that. But ultimately, bad boys generally don't create strong traditional families, for that ideally you need full package of the image of God and to be aware of all those traits to create a stable traditional family. Needless to say, you need wisdom to make money and not be dumpster diving. You need to be ruler of your family to prevent stupid nonsense like daughters wasting their precious virginity in drunk one night stand. You need to dispense judgement and chastise children correctly so they would get a sense of righteousness in a family. You also need to be in control of your woman so she would be happy, otherwise she will not consider you the lord worthy of worship and might run away to other men.

And the reward is the offspring, like promised to Abraham:

And I will make thy seed as the dust of the earth: so that if a man can number the dust of the earth, then shall thy seed also be numbered. - Genesis 13:16

As arrows are in the hand of a mighty man; so are children of the youth. Happy is the man that hath his quiver full of them: they shall not be ashamed, but they shall speak with the enemies in the gate. - Psalms 127:4-5


The entire red pill movement is based on what they say treating women like crap. The underlying reason is not that women need to be treated like crap all the time. The underlying reason why women are sexually attracted to that is that women know they can't be right, they are not as blessed in wisdom and understanding as men and they seek a man who can set them straight. A man determines rules for a woman, they can be evil or good but the women will follow them because they are wired to respond to judgement and shaming from her lord. Just like man ought to be drawn to worship someone who knows everything and understands everything, like the LORD.

The hatred of the image of God in the world

Now, the leftist cucks, the vast majority of the contingent of the software developers today despise the LORD and the image of God. They despise that women are not equal to men. They despise masculine figures that act like God, with might and judgement, i.e. Donald Trump. These faglets also despise the gym and cannot admit that women are attracted to strong muscular men. These faglets also despise wisdom and have none, are ever learning about worthless concepts but cannot come the truth, reality of the pattern and masculine and feminine planes:

Ever learning, and never able to come to the knowledge of the truth. - 2 Timothy 3:7

These are the fagglets who put rings on fingers of used up sluts when they squander all their youth and beauty on men who respect them the least. These are the cucks whose wifes cheat on them and they're the vast majority of the 30% of fathers globally who raise not their own children. That's why I'm so harsh to these people. These people are ignorant turd flinging monkeys. They cannot comprehend nor appreciate the divine design and its perfection. They neither are capable of engineering anything decent themselves neither they can teach others the correct way. Look at all the overengineered worthless garbage that is in the landscape of CNCF https://landscape.cncf.io/ . Hundreds if not thousands of projects of absolute insanity, where 99% of problems could be solved in a single codebase with the pattern. Every so few years new fads come and leave, like microservices, kubernetes, golang etc. that, once people practically find out that stuff doesn't work this way either, after wasting millions of dollars in overengineered designs they need to either hire hundreds more engineers to maintain all that mess or they have to rewrite everything from scratch again. There is an utter confusion of how to build anything if you don't have experience and greenhorn devs will pick the newest hottest "state of the art" overcomplicated garbage to achieve their goals. And the contractors are living in the infinite gold mine of overcomplicated unnecessary work.


There can only be two divisions, good vs evil. There are only two directions, say, being weak and being strong. Wise and unwise. Strict rules and discipline or a mess. Typesafety or dynamic typing. To go towards good, people need to strive to move to the good direction. Men ought want to be strong, wise and disciplined. Women ought to want to be chaste and submissive. Anything that goes against that is evil and ought to be shamed and scorned to maintain good. Get understanding and hate every false way.

Through thy precepts I get understanding: therefore I hate every false way. - Psalms 119:104
#28
Today, I'll explain from another angle why the pattern and meta executable will always be greater than doing things by hand.

I love to use real life analogies. They are amazing, you can explain very advanced concepts using terms every day Joe can understand. After all, every decent system relies on universal principles. For instance, father of the household is like God to his family (why God calls himself Father? So we'd understand him with earthly analogies of what father is?). Country with king is also same as God and his followers. Every natural divine system follows principle of masculine, godlike control plane and feminine, submissive plane that simply enjoys the fruits godlike control plane that thought everything out logically to be used.

Leftist cucks, since they went without divine wisdom in engineering for a long time, only produced abominations ever since by deviating from this truth. Well, not necessarily deviating, but rather being ignorant and never aware of that. Relying on their own pathetic weasel understanding, producing needless complexity everywhere without end and extreme toils and hard work for future generations to come maintaining that garbage.

What you have now in typical software development world, if you need a database, someone deploys database and you use it. If you need queues, someone deploys kafka. You need them working together? You deploy some other garbage. There is simply no system to connect them all together, so that all would be working towards the same purpose.

Unless, there is a divine all knowing meta executable from the pattern that knows about all these relationships and enforces them ;)

If you see the whole, not just parts separately, you can go wild. You can build extremely sophisticated abstractions that would otherwise be impossible. For instance, you can make sure database is reflected in clickhouse from postgres. Someone might say, "wait, doesn't debezium stack do that?". Yes it does, if you provide schema registries and all the dynamic garbage. Debezium stack will never check beforehand if the target table in clickhouse has the same schema is source postgres table. All this is left to a poor developer along with myriad of other runtime problems he will encounter in his lifetime.

You know what can easily check if such abstractions are valid? A meta executable that knows the following:

  • What kafka clusters are running
  • What databases and what tables are declared in postgres
  • What databases and what tables are declared in clickhouse
Having all this information in this place we can easily check the following:

  • Postgres and clickhouse and wanted kafka instances exist
  • Postgres schema matches clickhouse schema
And checking that we can provide a perfect error message, that says exactly what's wrong, that schemas don't match, instead of finding out about that somewhere in the logs in production ;)

Not only that, we can generate the following:

  • Any remap from any row with ends up in clickhouse
  • Possibly remap to another typesafe kafka topic with custom typesafe code
  • Maybe use clickhouse materialized views to generate another realtime stream to kafka topic

Possibilities are endless, because we have all information about everything as data in meta executable. With this approach we have a full body at our control, a body, that can use all of its muscles to perform single coordinated action, like striking a blow with a sword, do a pushup or dance ballet. The choice is ours. Did you notice, how seamlessly our bodies, under normal conditions, can make all the muscles perform unified action according to one mind? Or how unified the army marches? This is a divine pattern - everything is connected and coordinated to perform the same purpose.

The bodies of software development today work like this. An arm gets told to scratch the ass. The arm goes up to the ass and sees the ass is sitting. The arm goes to head to ask permission for ass to stand up. Permission granted, ass raises up. Hand finally goes to ass and scratches it. Imagine how bizarre this looks in your head - this is not normal. This is nonsense, paganism. All of these problems are trivial to resolve for a greenhorn developer if all the systems are declared as data in one place. Yet they create all sorts of nonsense these days, dynamic schema registration, separate schema evolutions in database, separate applications written to map stream to stream with no typesafe enforcement. Its extremely limited, because you inevitably stumble into the dreaded sentence "I don't know what schemas are in that and that subsystem, so, I'll use generic json and burden CPU with needless generic json parsing".

Things could be so much better than that. As I wrote in assumptions post, the more assumptions you make, the simpler your solutions will be. And the ONLY WAY to make these assumptions is to have all the data about your infrastructure in one place. Just like kingdom cannot be divided and stand, so is information about your entire project. If it is divided, there are bound to be conflicts and confusions. But if it is united, all in one place, and checked for consistency on a very high level - such project can stand.

And if a kingdom be divided against itself, that kingdom cannot stand. - Mark 3:24

Conclusion: software development paganism, division and confusion is cancer and causes huge complexities in software projects. Divided projects without strong interconnections between them are summed parts. Whole, where all your infrastructure is defined as data in one place is greater than sum of its parts and opens unlimited possibilities for correctness checking and code generation.
#29
Today, most software developers are delusional, low testosterone, pussified people. These people cannot hear anything negative about their programming language, their beliefs, design patterns and so on. Doe-eyed cucks today usually are more keen to accept an incompetent pussified faglet as a leader, that has no deep technical skill, no decent logical insights, but is rather smiling and diplomatic and would never hurt anyones feelings.

This is disaster, because every improvement to our modern society was made when someone noticed that current things are subpar and need to be changed. One of the reason for this mental illness in sillicon valley is probably due to fact that nobody is measuring productivity of separate developers and nobody asks why some developer does things 10x while others linger for weeks on the same task. Even if reason is given, if you're around ruby monkeys, and you say "typesafety is amazing, I don't need to spend that much time debugging code and can focus on the problem", the usual knee-jerk irrational reaction is "we write tests so we have the same". No you don't, outcome of two different methodologies can never be equal as I've discussed in world of inequality thesis.

So you get a huge array of different methodologies of doing things wrong. In media it is beneficial, for instance, if you were youtube, you'll get bigger audience by servicing 100 ways to do things wrong. It's just more content. The more content you have, the more imaginary value. But, if you go by the way of simplicity, saying there is one right way to do things and the rest are a waste of time (like I teach with the pattern), then, all of a sudden you become enemy of all the teachers that teach 100 wrong ways to do things, with your clearly superior solution.

You see, natural gas companies don't fight oil companies, they both can make a living selling fuel. What they all unify together against is countless of inventors, like Steven Meyer, who invented water ran car https://www.youtube.com/watch?v=staL1wr07Sg . These companies quickly smell that they can become obsolete in an eye blink and kill or persecute the inventors.

Same with programming languages, if you state that python/ruby/javascript are unproductive, unmaintainable abominations due to dynamic typing and nobody should use them to implement any reasonably complex project, you become an enemy. All the false teachers, that teach nonsense about design patterns, domain driven development or what not, quickly become enraged when any other superior way to do things is shown. Nevertheless, people who seek wisdom and love righteousness will carry the truth in their heart.

This is the same thing when Jesus said, the world would not accept you.

If ye were of the world, the world would love his own: but because ye are not of the world, but I have chosen you out of the world, therefore the world hateth you. - John 15:19

Any idiot can go with the flow, like microservices are the thing, like kubernetes is overtaking the industry, like golang, despite being an utter trash is getting all the tooling. Any idiot can nod head and agree with anyone else. But it takes wisdom, wisdom of the minority who can judge in truth. Who understand consequences and implications. Who took the beating and wrote hundreds of thousands of lines of code and know what works and what doesn't. Any greenhorn that just comes out of university has his mind instantly polluted with complicated patterns and impractical technology. People in university are taught that there is java and that there is javascript, but there is no judgement that comes with it, that one is typesafe, and can be used to write millions of lines of code, and other is for weekend projects.

Are people expected to come to such judgements on their own? But how can they, without hingsight? So, they are doomed to wonder from cult to cult, without having a right way. They may do java with xmls for a few years in a bank, then, other cultist introduces them to ruby and how little code they need to write. Ruby cultist doesn't introduce them to problems that their production rails app is full of undefined method errors, which could be checked by a compiler in statically typed language. They don't say to them that they waste half their time writing unit tests. Then the new cultist rubyist either gets stuck in ruby, and assumes it is normal to push few other commits to production to fix undefined variable errors and accepts this as a norm. Or a more astute one will think "statically typed languages don't have this problem". Even if they do figure it out, they wasted years of precious time getting there and they will never get it back.

Good thing such things as this forum exist where with the pattern you get to the ultimate end solution that can solve everything ;)

But you see, I'm the asshole. I'm the bad guy. I'm in the wrong, because I say dynamically typed languages are a waste of time. I'm in the wrong because I can offer solutions of how to have typesafe SQL and knowing it will work before production. I'm the asshole, because I say current infra management tools like chef are an abomination that are very fragile and waste everyone's time. I'm the asshole because I say yamls are worthless and can be replaced with typesafe programming language to encode any configuration you want. I'm the asshole because I say what most other people know about software development is overcomplicated nonsense. I'm the asshole because I know how to do things much more productively, saving everyones time and money and likely hiring much fewer devs to achieve the same result.

Well, as history shown time and time again, even if you supress solutions to problems long enough, they'll come out sooner or later to the surface.

For nothing is secret, that shall not be made manifest; neither any thing hid, that shall not be known and come abroad. - Luke 8:17

And when everyone finally realizes, all this time emperor had no clothes it will be like the dam breaking upon everyone. But some people just hate reality and hate the truth - this is the way of the world and it will always be.
#30
Main Thesis / Part 6 - Assumptions
June 04, 2021, 10:19:20 AM
I want to talk about simple fact that permeates through all of life. And it is illustrated below.



The more assumptions you make the simpler code can be. The less assumptions you have the more complex code is.

Consider two solutions to a function that reads a file to string of bytes. For me such function is a phenomenon. Many programming languages, like Java, did not have a single function that can read all file contents to byte string back in the day. You have to use apache commons jar.

Why is that? Such functions make development much easier. Many answers in stack overflow suggest creating buffered reader and then spinning a loop https://stackoverflow.com/questions/731365/reading-and-displaying-data-from-a-txt-file , why all this complexity?

The knee jerk reaction to this is the usual saying "well, what if the file is 100 exabytes, you couldn't fit such file in memory". That is true, we could not read exabyte file into memory. But how often do we do that? If we need to manage that much data in a single file, SQLite is a better option anyway. But the files that we usually need to read, say, source file or a secret, or a json file - they are usually tiny. Why do we complicate 99% of the cases where the file is indeed small and function to read it can be super simple with a needless abstraction of reading it with stream just in case it is exabytes?

Reading a stream is a generic (there is that evil word again), low level, complex solution. If solution is specific to small files then it is very simple - just a single function, read all file into memory and be done. Another thing, Java cannot assume file is ASCII. So, you have to convert that certain stream of bytes via UTF-8 encoding into java's string, which is ironically two bytes. What if we eliminate that assumption too? UTF-8 is universally supported format, we can make our read file to string function simpler again.

You see how we made two assumptions and our problem and solution became simpler?

Infra management

Current software developers world is full of such complexities due to lack of assumptions (lack of information). For instance, in infra management tool Chef, which I utterly disrespect and I think every company is better off writing their own in OCaml, there are folders for files and templates. Every chef cookbook has a default folder, then could have windows/linux/solaris and what not folder for specific templates to that platform. What if company only uses specific version of linux? All that becomes useless complexity. Since chef is a generic, used up, smurfed out, whorish solution that kinda does a crappy job at everything, which is jack of all trades, master of none, it also brings all the complexity of every hipster company into one project for everyone to bear. Same with kubernetes complexity.

I heard arguments saying "Well, would you rewrite chef yourself? You know how many man hours were put into that thing?". Yes, I would. I would implement the part I need below



This is the assumption I would make in the infrastructure:

1. Use certain version of debian
2. All nodes must have docker installed
3. Any component that is needed, postgres/kafka/clickhouse/prometheus - you name it, is installed via docker

What I don't need to implement from chef this way in my OCaml infra management tool (which I did implement for myself by the way):
1. Support for any other OS but linux debian
2. Adding cerain debian package repositories of, say, newest postgres - just run that in docker
3. All the kitchen insanity, I'd turn component configs into data and analyze the data of configs instead using the pattern

What I'd improve upon:
1. Maybe implement iptables if I wanted to restrict who talks with that
2. Typesafe roles, unlike raw jsons that chef uses, with which it is very easy to make a typo that will end up in production
3. Implement typesafe, no nulls allowed function of how to install generated infra file to filesystem
4. Implement systemd functions, to, say, install certain systemd unit for things like vector to collect logs

Infra management tools ought to be a child's play. They have nothing inherently complex about them, just generate some files and write them into filesystem, maybe restart systemd service or spinup docker container with mounted directories. These things will never be as complex as, say, doing 3d vector maths for a game or development of relational database or programming language. Yet people who write infra management tools, since their task is so simple and they want to pretend that its not, they have to needlessly complicate things for themselves by making what they write as generic as possible for every use case under the sun, hence dragging along huge complexities to otherwise very simple software.

You see, you don't get the best and most ingenious people to work on tools that write some files to filesystem, spin up few systemd services and run a few docker containers. Just like we discussed in earlier post, that nothing is equal, it is also not equal the brilliance of developers that work on different problems in the industry. Most brilliant devs are likely working on games, databases, programming languages, theorem provers and so on. This is where the smartest minds are at. And in web development you have much lower quality people, where brilliance is not a concern for generating web pages while querying database. Same with frontend and same with infra management tools.

Just the fact, that there are really only three choices infra management today, namely Chef, Puppet and Ansible, and two of them are developed in Ruby, Ansible in python - that speaks volumes of how lowly esteemed and underdeveloped and uncompetitive that niche is. Infra management is the Africa of software development. Writing code, that could potentially bring down hundreds of machines in production to a standstill, with a language, which you MUST RUN in order to see if your code works - that's a recipe for disaster. And proponents of such tools are sufferers of a deep mental disorder. Usage of raw jsons to describe roles, where you can make any sort of mistake, instead of typesafe struct in programming language like OCaml, where illegal cases are mostly irrepresentable, is absurd.

Conclusions

The more assumptions you make for specific solution you want to implement, the simpler your solution will be. You don't always have to fear "such and such pile of crap is developed 10 years - we could not implement it ourselves" statement. Sure, I wouldn't implement something like Postgres on my own, it is a very powerful database that can cover anything you need. But such weekend projects as Chef, that don't really do anything at all besides writing files to filesystem, it is much easier to just implement trivial solution by yourself ( with, say, rock solid typesafety of OCaml and the pattern ;) ) and only implement the features that you actually need. And overall you will have very simple specific code that will be easy to maintain and will not have all the Chef complexity in it too.