My Coding Cult

In this post I will reveal most valuable programming pattern I know. It's extremely solid, versatile and opens limitless possibilities. I've discovered this independently myself, because I knew that is the way the LORD had to create everything. There is no other way to do things (everything before this is a wrong way), and it will be clear why. It is so deep, its core goes to the foundation of both sexes, male and female, masculinity and femininity and what and why that is at all.

Show me any programming language feature you know without which you cannot live and it all pales in comparison to this pattern. It is irrelevant and useless and has no meaning compared to this. In fact, I can do any language feature you ever knew with this pattern while host programming language only needs basic features.

You don't even need to practice it, it will be obvious - there is nothing that you will not know about this pattern that I know just reading this post.

If you only read single post in this forum, it better be this post.

Okay, hype speech is over, let's go.

Represent abstractions as data

What does that mean? Sounds vague marketing sales bullshit, let me explain it very simply. Instead of connecting to the database and sending the query from code, we define query as data in a program that will generate typesafe call to a function. Here is OCaml example of what I mean:

let res = execute_write ~params:[|string_of_int voting_id|] pg "
  UPDATE voting_session
  SET signatures_count = signatures_count + 1
  WHERE voting_id = $1
  AND NOT is_scheduled
  AND NOW() < petition_deadline
  " in

This code block executes SQL query in Postgres and you pass parameters and convert them to strings by hand.

Every single time you write such block you are risking of making a mistake. Not nice. Ruby monkey answer to this? "Just write tests!". Oh yes, If I was idiot Ruby monkey I'd see no issue with writing tests for every single most trivial code block for everything.

Here is the alternative:

let db_queries = [
...

  mk_db_mutator
    FtVoting
    "increment_voting_signatures_count"
    {|
    -- voting_id:15
    UPDATE voting_session
    SET signatures_count = signatures_count + 1
    WHERE voting_id = <voting_id:int>
    AND NOT is_scheduled
    AND NOW() < petition_deadline
    |};
...
]

I defined exact same query as data in a list of other queries. mk_db_mutator function just returns struct of this query.

This data resides in a meta executable that generates OCaml code for a typesafe function:

let dbm_increment_voting_signatures_count (pg: Postgresql.connection) ~(voting_id: int) =
  incr dbm_increment_voting_signatures_count_init_count;
  let timestamp = now () in
  send_db_query_record ~timestamp ~query_label:"dbm_increment_voting_signatures_count" ~duration:(-1.0) ~rows_affected:0 ~rows_returned:(-1) ~arguments_keys:["voting_id"] ~arguments_values
:[string_of_int voting_id];
  let res = execute_write ~params:[|string_of_int voting_id|] pg "
    UPDATE voting_session
    SET signatures_count = signatures_count + 1
    WHERE voting_id = $1
    AND NOT is_scheduled
    AND NOW() < petition_deadline
    " in
  let duration = now () -. timestamp in
  let rows_affected = int_of_string_opt res.result#cmd_tuples |> Option.value ~default:(-1) in
  incr dbm_increment_voting_signatures_count_success_count;
  incr_float dbm_increment_voting_signatures_count_duration_sum duration;
  send_db_query_record ~timestamp ~query_label:"dbm_increment_voting_signatures_count" ~duration ~rows_affected ~rows_returned:0 ~arguments_keys:["voting_id"] ~arguments_values:[string_of_int voting_id];
  res

So, compilation of codebase goes like this:
1. Meta executable runs, performs all checks, and if everything is okay emits code
2. The rest of my codebase compiles, and uses typesafe functions

Not a big deal? Okay, let's go through all the things that one small query has checked when it is defined as data:

1. It has to work, mutate data in the table, it is run with example data against test database in mind
2. We check that it does mutate a row
3. <voting_id:int> is substituted and added to a typesafe function, I can add arguments in any places I want
4. Code is generated to save EVERY SINGLE QUERY in Clickhouse WITH THEIR ARGUMENTS in a NICE MAP. I have every single query ever executed against the database, well compressed, with their parameter names and values in Clickhouse.
5. Prometheus global variables are generated and incremented and automatically exposed to prometheus.
6. Every single query is assigned to some feature enum, so, if I delete features logically I have to go through all the code and see where it is used
7. If I change query signature, I will have to refactor code where I use this function.

Imagine, I just write very little code, of defining query as data and all this code is generated and all these things happen on autopilot. I have confidence that queries will work and things work 99% of the time. I barely write any tests either.

My mistake factor on producing invalid queries is now drastically reduced, and I can write very complex queries and know things will mostly work.

Imagine you're at work, and someone says, we have to label and monitor all queries in the database. Find places where we query database and make sure all is tracked.

Well, you could be lucky, attach to the framework, but what if you have lots of code that just uses SQL library which has no concept of tracking? Do you track in database?

Here, what I did, SINCE I HAVE ALL QUERIES DEFINED AS DATA, I added few lines of code to:
1. generate prometheus global variables
2. generate function that dumps all these metrics to prometheus string
3. few lines add increment to global variables in their special global variables

Wow, so simple anyone can do it. And you can do anything you'd want when you generate code from data. Sooooooooo simple!


Database tables

I also define them as data. Not as raw SQL.

let db_tables = [
  ...

  mk_db_table
    11
    "chat_message_likes"
    [
      mk_field 1 "message_id" Int ~index:true;
      mk_field 2 "user_id" Int ~index:true;
      mk_field 3 "time_liked" Timestamp ~default:"CURRENT_TIMESTAMP" ~index:true;
    ]
    ~uniq_constraints:["message_id"; "user_id"]
    ~test_data:[
      ["112"; "10"; "NOW()"];
      ["113"; "12"; "NOW()"];
    ]
  ;

  ...
]

And the RAW SQL is just generated.

mk_field 1 - field number, is unique identifier for field that does not figure in SQL, so, I could rename fields in SQL if I want to.

So, I can rename a column, keep same number, and migration is generated that renames column.

Guess what? If I change schema in the data my queries break, I have to fix that. Everything has to work together.

Which is nicer:

- Having raw SQL to define schemas, maybe parse SQL Ast and try figure out what it does, and write migrations by hand also
- Define schema as data and just generate all migrations with simple code generation?

I choose second, and it paid dividends time and time and time and time again.

I have single test that checks that migrated schema from previous version must be same as freshly initialized, so, I can tolerate errors in migration function, because they are detected early and never reach production.

What I could also do, since I have schema as data:
- Generate code for automatic monitoring of int overflows
- Say that table has logical unique constraint, but don't enforce it because of performance and generate code to monitor it
- Maybe flag some tables that need to be mirrored to Clickhouse for better performance and generate Clickhouse code that will keep refreshing table in Clickhouse

Possibilities of this approach to solve problems are endless.

REST endpoints

Rest endpoints? I have them as data. Here's an example:

let rest_endpoints = [
  ...

  mk_rest_endpoint
    FtForum
    "/api/post_to_chat_room"
    LoggedInUser
    (CustomAction "post_to_chat");

  ...
]

As you can see, I even have enum, whether endpoint is accessible to logged in user or to outsider.

What is custom action you ask? I have all custom actions defined as data:

let custom_actions = [
  ...

  mk_custom_action
    FtForum
    ~return_type:"post_to_chat_return_result"
    "post_to_chat"
    [
      "chat_room_id", Int;
      "contents", String;
    ];

  ...
]

They have type signatures and arguments. What type signature the rest endpoint has depends on the custom action. For instance, now rest endpoint receives json with body of `chat_room_id` and `contents` - it comes from custom action. It could database query mentioned earlier instead, and rest endpoint would accept different argument.

How do I call this rest endpoint? I'm glad you asked, you just use typesafe OCaml function that under the hood gets results of this custom action and you only provide typesafe function. In fact, we receive defined return type:

type post_to_chat_return_result =
  | CAR_PostMsg_MessageTooLong
  | CAR_PostMsg_EmptyMessage
  | CAR_PostMsg_UserDoesntBelongToRoom
  | CAR_PostMsg_Ok
[@@deriving yojson]

No json parsing by hand, I just call generated function in the frontend:

rest_post_post_to_chat { chat_room_id = chat_room.room_id; contents = contents } (fun res ->
    match res with
    | CAR_PostMsg_Ok -> (
        materialize_toast "Hoooray, posted!"
    )
    | CAR_PostMsg_MessageTooLong -> (
        materialize_toast "Message to long"
    )
    | CAR_PostMsg_EmptyMessage -> (
        materialize_toast "Message empty"
    )
    | CAR_PostMsg_UserDoesntBelongToRoom -> (
        materialize_toast "You do not belong to this chat room"
    )
)

If I add more enum outcomes in the backend I MUST also fix the frontend to account for such case.

I don't deal with raw json. I don't deal with null pointer exceptions. Everything is typesafe and usually works first time and there are no surprises. I don't understand why people would ever want to code frontend in plain javascript, suicide!

Let's look at all code that was generated to serve/call this rest endpoint:


  post "/api/post_to_chat_room" begin fun req ->
    wauth_succ "backend_rest_ca_post_to_chat" req (fun session ->
      App.string_of_body_exn req |> Lwt.map (fun json ->
        match Yojson.Safe.from_string json |> gen_type_7557852441389680360_of_yojson with
        | Ok args -> (
            let with_connection = with_connection ~ray_id:(get_ray_id_from_request req) in
            let api = mk_custom_action_auth_api "post_to_chat" with_connection session req.request in
            let result = ca_post_to_chat api (args.chat_room_id) (args.contents) in
            result |> post_to_chat_return_result_to_yojson |> Yojson.Safe.to_string |> respond_json_str_lwt
          )
        | Error err -> respond_json_str_lwt "PHEIL"
      )
    )
  end;


All of the tricky stuff, about authenticating, parsing right type, right arguments, and returning type as json is generated.
The function that is not generated is ca_post_to_chat and is a perfectly typesafe function that just receives appropriate labeled arguments:


let ca_post_to_chat ~(api: custom_action_auth_api) ~(chat_room_id: int) ~(contents: string) =
  if String.length contents > 1000 then (
    CAR_PostMsg_MessageTooLong
  ) else if String.is_empty contents then (
    CAR_PostMsg_EmptyMessage
  ) else (
    with_connection "post_message_to_chat" (fun c ->
        let res = dbq_does_user_belong_to_chat_room c
            ~user_id:api.session_data.user_id ~room_id:chat_room_id in
        if res.(0).user_belongs > 0 then (
          dbm_post_to_chat_room c
            ~user_id:api.session_data.user_id
            ~chat_room_id ~contents |> ensure_pg_ok;
          CAR_PostMsg_Ok
        ) else (
          CAR_PostMsg_UserDoesntBelongToRoom
        )
      )
  )


As you can see, I don't deal with anything that is HTTP related, I declare rest endpoint, state custom action with its arguments, its return type (that will be perfectly reflected in frontend, bye bye faggitty HTTP return error codes), and I have to provide typesafe function to fulfill that contract and if anything is wrong, compiler will error out and I cannot ship to production.


This is generated frontend code, so I could call this rest endpoint just as any other function with result callback:

let rest_post_post_to_chat (args: gen_type_7557852441389680360) with_result =
  let url_var = Printf.sprintf "/api/post_to_chat_room" in
  let body = gen_type_7557852441389680360_to_yojson args |> Yojson.Safe.to_string in
  post_ajax_url_json_wcallback url_var "frontend_ca_post_to_chat" body (fun str_res ->
      Yojson.Safe.from_string str_res |> post_to_chat_return_result_of_yojson |>
      function
      | Ok result -> (
          with_result result
        )
      | Error err -> (
          print_endline "Cannot deserialize custom action"
        )
    )


I don't deal with knowing what is the right URL to call this - all generated from same source, and both perfectly match!

Plus, I make these checks of the top of my head:
1. Custom action must exist to be attached to REST endpoint
2. You cannot call REST endpoint with non existing arguments, because same generated type (without nulls) is both in frontend and backend, and same serialization function parses them
3. There cannot be duplicate REST endpoints, even if I have thousands of them (and they grow very fast, because adding many of them is now trivial due to code generation) duplicate paths will be detected
4. It is generated that every REST endpoint with its label is logged and monitored in prometheus automatically.
5. If I break query in the backend, and type signatures change I must refactor the frontend too


Thoughts

What framework offers you that? Because everything I see is a joke compared to what I do myself. And that is for the very simple fact:

DEFINE YOUR PROBLEMS AS DATA

If I had to write separate calls to rest endpoints and separate calls in backends everything would become a mess very quickly. If I had no typesafety, things would become a mess very quickly.

TYPESAFETY IS A MUST IF YOU WANT TO DO SUCH THIGS.

I rip for everyone I meet about typesafety. Most people can't understand it because they never tried to do something as powerful as this pattern, so, they think its okay to push non working code, someone notices and push some other garbage code to fix first garbage code. But if you want to do things like this pattern, try doing it with NodeJS or Ruby and we'll see how much you'll last. You'll start drowning in infinite amount of runtime errors very quickly. Hence, we have to use statically typed language, that will check our generated code integrity or these exploits become practically impossible.

Now I hope everyone understands why I don't take any dynamically typed scripting languages seriously, be it python or javascript or ruby. I don't need them. I do much more powerful things, with much greater correctness and typesafety. I barely spend time debugging code, because it just usually works. Ruby developer has to write 2x code, implementation and tests because he has no choice. I write 90%+ percent implementation code on rock solid abstractions and very little tests, for pesky things like making sure regex is parsing what I expect. I couldn't imagine writing trivial tests for every REST endpoint, or database query - what a waste of time. Things like that work from the first time in vast majority of the cases and I can focus on the application.

Sure, I wish some features were present in OCaml that ruby has, like decent string interpolation, but again, I did not find it issue enough yet, but, if I will, I will just make typesafe template abstraction in meta executable.

Who forbids me of defining templates like that? This code does not exist, but I'm just fantasizing:


let templates = [
  ...
  mk_template "some_template_ini" {|
    ip_address = <the_ip:string>

    open_port = <port:int>

    <iterate peers { peer_port: int, peer_ip: string }
    connection = <peer_ip>:<peer_port>
    <iterate end>
  |};
  ...
];


And then just generating typesafe function with all these arguments:

let template_string = tplt_some_template_ini ~the_ip:"127.0.0.1" ~port:8080 ~peers:[{ peer_port = 1234; peer_ip = "1.1.1.1" }] in
...


What forbids me from doing that? Hence, I don't care about programming language features.

I only need:
1. Decent typesafety (like OCaml and Rust provides, Java with random null pointers is way too weak)
2. Meta executable with my domain problem defined as data

Other benefits of generating code yourself in meta executable:
1. Your IDE will be grateful, it can parse trivial generated language code, instead of having to support very complex language features
2. You have a reference point of how generated code looks, so, you don't need to wonder, like in Lisp what macros have done and why stuff doesn't work
3. Ultimate flexibility to have cross context consistency, imagine, generating language bindings to your rest endpoints for huge amount of different languages
4. Unlike C/Rust/Lisp macros, which are very weak and just do local context this allows you to evaluate global context of everything working together from all different sides (you can check that for instance, a queue exists but is not used by anyone)
5. Since your problem is defined as immutable data, you can test your data very fast, mostly in memory consistency tests for your domain (avoiding pesky integration tests that require spinning up machines)

This is so simple anyone could do this. You don't need PhD's (people in university are overrated anyway), only common sense. If I taught a greenhorn coder this, he would probably beat the living crap out of 99% seasoned developers in productivity and correctness.

That's it. Not interested in Rust macros, Lisp macros, Rebol, Red, Scala, Go, V or any of 100 languages that will come out next year and forever. I can do everything on my own and generate anything I ever wanted. Practical applications of this pattern are endless and productivity gains are immeasurable. No wonder I consider everyone else around me an idiot now :/

I will also explain the logical reasoning and philosophy behind this. We will touch sexes, why even women are attracted to a certain behavior in a man, feminine and masculine planes and all creation.

I wanna speak about programming languages I use. Tried lots of crap over the years and I found a few gems that are my go to tools and I don't need anything else. I have access to abstractions way more powerful than any programming language gives you out of the box (and sadly, no one I know uses this trick, I had to come up with that myself).

I wan't to speak about languages I use for everything I do, and I don't need anything else:

Languages I use

OCaml

The best language I've ever used, by its expressional power (which is limited, but can be unlimited). The rock solidity of type safety and type inference makes this indispensable. Single thread C like performance is also great, hence, you can develop websites, or anything really, without wasting too much CPU.

This is a variable in OCaml:

let x = 7

This is a square function in OCaml:
let square arg = arg * arg

This is how to square list of elements in ocaml:

List.map [1; 2; 3] ~f:(fun i -> i * i) ;; will yield [1; 4; 9]

As you can see, very expressive, much more expressive than C++. But most of all: TYPESAFE. You write script-like programs but they have to compile to be shipped. All types in examples are inferred, if they are not inferred you can explicitly specify them.

Usually, scripting languages are garbage. You find out if they work only during runtime. Here, 90% of trivial errors are caught before you ship anything, needless to say, a huge time save.

If project doesn't need to squeeze every drop of performance from CPU (90%+ projects are like that), choosing OCaml is no brainer and allows writing tens of thousands of lines of code, mostly without tests, that usually just works. I can't imagine any other language that even would come close.

Rust

Sweed delivery from pain points of C++. No more segfaults. If something is performance critical, picking Rust is a no brainer. Already there are 10x more libraries for Rust than C++. Rust will never be as terse as OCaml, being a low level language, but with a good reason. We have to get to low level to, and be very explicit about telling CPU what we wan't to do. Square function in Rust:

fn square(num: i64) -> i64 {
  num * num
}

As you can see, quite more verbosity, and mandatory type annotations which we didn't have for OCaml, but hey, if you want performance, this is the sacrifice I'm willing to make.

That's it

These are all the languages I'd ever use in production and with which I develop everything, from compilers, to web services, to DSL, to metrics systems - anything.

I DO NOT USE ANYTHING ELSE.

Hence, I know only two languages, which allow me to do any abstractions I can think of and I laugh when I see other languages introduce new features (because I already have something more superior than any language feature, and these are not things that come out of the box of OCaml nor Rust)

---

Languages I'd never use

Now, let's roast the languages I don't care about (even if I know them) and why I would never ever use or need any of them

C++

Rust did everything C++ failed at 100x better. From macros to package managers. If you're starting a new C++ project in 2021 you're seriously screwed in the head.

Go

A pathetic excuse of a language. Neither as fast as Rust, neither as expressive as OCaml. No parameterized typing (pathetic). No full type inference (I don't roast Rust for that because it is fast).

Pathetic error handling:

f, err := os.Open("filename.ext")
if err != nil {
    log.Fatal(err)
}

Are you serious? Not even Result<Success, Error> monad? Not even Some(..) None option? Check for nil? Go kill yourselves.

The only decent feature is portable executables.

If this is the best google and rob pike can come up with then I can't help but state that they're bunch of idiots.

Javascript

For frontend, I use js_of_ocaml which compiles to javascript monstrosity. But I do have to sometimes write itsy little bitsy of javascript, for instance, if you have javascript library, I write bindings to use it frm typesafe OCaml, I make sure they work and just write OCaml and enjoy typesafety. Also, if I need to access rest endpoints they are typesafe too, I don't parse jsons by hand.

I would never use javascript because it is not typesafe. I don't want to check spelling errors by writing unit test for every most trivial function in the world. I want to write code that works from the first time 90% of the time and barely have any tests, except for exceptional circumstances. This goes for any dynamically typed language - run from them.

Python

Non typesafe, all the drawbacks of javascript. There are circumstances when I'd write up to 100 lines of throwaway scripts with python, as it is already installed in machine, or python has some library I need, but again, I wrap those libraries in typesafe way and enjoy them from OCaml without any drawbacks of dynamic typing.

Ruby

Would never use, no idea why anyone would use it. Does not come preinstalled, is not typesafe, either write 2x code, 1x for code, 1x for tests, and even then type safety errors creep into production. Performance is also horrible. Community is generally very low-quality people, that know something horrible, like java, they learned ruby and then they think that it is all there is to the world. No OCaml developer I've ever met or talked with gives a tiny rats behind about ruby, because from OCaml standpoint it's pathetic.

Java/JVM

In general, avoid anything with JVM. JVM people tend to produce very complex projects, avoid 99% stuff from apache software foundation. For every pathetic pile of crap ASF puts out eventually there comes a simpler, more easily maintained analog.

I will not post syntax of Java (Scala is a decent attempt to save sinking JVM ship) not to rape anyone's eyes, but boy is it verbose. And boy is it error prone. And boy there are many null pointer exceptions in production. And if you want to save money on hosting or hardware, OCaml executables serving websites run up to 32MB of usage (I'm generously overestimating), if you deploy any JVM pile of crap, in the most conservative setups be prepared to sacrifice at least 128MB of RAM. Reality is much more grievious, JVM process that uses 1GB of RAM is fairly modest.

If you choose anything JVM based to develop your website solution, forget about running few cheap digital ocean boxes to serve your website (no problem with OCaml) until it gets popular and startup idea is justified. Be prepared to call devops team and pay them gazillions to build your private cloud with kubernetes instead, then, once you've wasted tens of thousands of dollars for servers and development, see your startup fail and all of that startup cost as a waste.

Why would anyone start with JVM? Bizarre.

Lisp/Clojure

There are lots of fanbois of lisp there, been there myself. Then one day I figured I'm spending 90% time debugging non typesafe code. And Paul Graham states it is the most powerful language, because of muh macros. Lisp macros to what I will reveal here and what I do with OCaml/Rust will be pathetic and pale in comparison. So, if we don't care about lisp macros, lisp becomes similar to other non-typesafe languages and which we already dismissed as garbage,

Muh favorite some other language

Sure, post something else, but 99% chances are, I don't care. It will never have feature which I cannot develop myself (sounds hard to believe, but when I explain it, it will be so simple, you'll just think "why didn't I think of this myself?"). In general, I'm not limited by any pesky language features, all I need is basic type safety and I'm set to build anything I want on top of that.

Sup bois. Your boi here, the cult leader of this forum.

As you can see, by a domain name, this forum is a hacking cult. And I don't mean it in a good way, like people here are cult-like passionate (even though I am passionate).

But I mean it in a way, I can't stand all the unproductive, low quality, doe-eyed leftist cuck software developers and their excuses today and hence I've created a cult of my own way, of my own productivity.

What this means, if I hear typycal BS you'd read on Quora, like there's no such thing as bad database, there's no such thing as bad programming language, everything is appropriate at its own due time, I'll ban your ass.

I deal enough with idiot developers in a daily life, I will not deal with them here.

I'm a douchebag, I love myself and I embrace it - I tell it like it is without any regard to your feelings.

There will be one idea of how to do things - that idea is mine (sorta, inspired by the LORD, my greatest teacher). You don't like that? Tough, this forum is my hacking cult, so feel free to get out.

Now, of course, reasonable people can ask questions of why things will are done certain way, and everything is divinely simplex, easily explained, rationally and logically, but the moment I smell faggotry or trolling - people will get banned.

Congratulations on reading my opinions, there are lots more to read and I hope you learn a lot. In fact, if you indulge in all the advice that I give in this cult, you'll be top 1% developer that thinks ahead, and will have amazing insights ahead of average, brainwashed, braindead, leftist cucks who will stay unproductive and miserable with their stupidity.