Eden DB Improvements 2

[CultLeader]

Sup guys, it's been a while you lurkers that read my posts but stay silent.

I'm working on Eden platform now, and basis of that is EdenDB, to which I make changes as I need them when working on eden platform. I've made quite a few improvements since last improvements post.

Ability to refer to other element's children through REF FOREIGN CHILD syntax

The need for this arose when I needed for abstractions, like NATS or PostgreSQL instances to specify on that volumes they reside.

Say this is a server table (simplified)

Code: [Select]

TABLE server {
  hostname TEXT PRIMARY KEY,
}It has volumes as children (simplified)

Code: [Select]

TABLE server_volume {
  volume_name TEXT PRIMARY KEY CHILD OF server,
  mountpoint TEXT,
}
So say you have server-a with defined volumes

Code: [Select]

DATA server(hostname) {
  server-a WITH server_volume {
    pgtest1, '/srv/volumes/pgtest1';
  };
  server-b WITH server_volume {
    pgtest1, '/srv/volumes/pgtest1';
  };
}
And we need to define a postgres instance of a single cluster (simplified from real schema)

Code: [Select]

TABLE db_deployment {
    deployment_name TEXT PRIMARY KEY,
}

TABLE db_deployment_instance {
    deployment_id INT PRIMARY KEY CHILD OF db_deployment,
    db_server REF FOREIGN CHILD server_volume,

    CHECK { deployment_id > 0 },
}
So now we say that we want our foo logical database, with three replicas defined, one will be master with patroni and other replica.

Code: [Select]

DATA STRUCT db_deployment [
  {
    deployment_name: foo WITH db_deployment_instance [
      {
        deployment_id: 1,
        db_server: server-a=>pgtest1,
      },
      {
        deployment_id: 2,
        db_server: server-b=>pgtest1,
      },
    ]
  }
]
Notice, how we refer to the volume of `server-a` and `server-b` where we want data to reside using `=>` operator, which means child element.
We could nest this arbitrarily deep.

So, by using foreign child syntax syntax we do two things:
1. Specify which server runs the database (parent of the volume)
2. Specify where data on server runs

And, of course, you cannot define non existing elements in EdenDB, so you cannot make a typo of specifying non existing server or volume (unlike in typical yaml hell)

Full gist is here https://gist.github.com/cultleader777/41210e58026a0bec29f7e014945e40b0

Refer to the child element with REF CHILD syntax

Another issue I had working on eden platform is that I wanted to refer to the child element from parent element.
Say, server has multiple network interfaces, how to we specify which interface exposes ssh for us to connect and provision server with?

Say this schema (simplified from original)

Code: [Select]

TABLE server {
  hostname TEXT PRIMARY KEY,
  ssh_interface REF CHILD network_interface,
}

TABLE network {
    network_name TEXT PRIMARY KEY,
    cidr TEXT,
}

TABLE network_interface {
    if_name TEXT PRIMARY KEY CHILD OF server,
    if_network REF network,
    if_ip TEXT,
    if_subnet_mask_cidr INT,
}
So, network interface is a child of server and we can refer to the child from the parent so that we know on which network interface we ssh to every machine.

Code: [Select]

DATA STRUCT network [
  {
    network_name: lan,
    cidr: '10.16.0.0/12',
  }
]

DATA server(hostname, ssh_interface) {
  server-a, eth0 WITH network_interface {
    eth0, lan, 10.17.0.10, 24;
  };
}
Also, we can refer to child of arbitrary depth with the `=>` syntax from parent.

Code: [Select]

TABLE existant_parent {
    some_key TEXT PRIMARY KEY,
    spec_child REF CHILD existant_child_2,
}

TABLE existant_child {
    some_child_key TEXT PRIMARY KEY CHILD OF existant_parent,
}

TABLE existant_child_2 {
    some_child_key_2 TEXT PRIMARY KEY CHILD OF existant_child,
}

DATA existant_parent {
    outer_val, inner_val=>henloz WITH existant_child {
        inner_val WITH existant_child_2 {
            henlo
        }
    }
}
As usual, EdenDB fails to compile if existing child elements don't exist.

Full example here https://gist.github.com/cultleader777/d4f26449d2814a30d6b34e55c5d19c76

Detached defaults with DETACHED DEFAULT syntax

I had an issue of working on Eden platform that if default is defined in the database schema then it can't be changed by the user.
This is because eden platform schema resides in one set of files and user defines data in his own set of files.

Say, server has hostname and belongs to tld in the schema file

Code: [Select]

TABLE server {
  hostname TEXT PRIMARY KEY,
  tld REF tld DETACHED DEFAULT,
  fqdn TEXT GENERATED AS { hostname .. "." .. tld },
}

TABLE tld {
  domain TEXT PRIMARY KEY,
}
Now, it would be not nice to specify to which TLD server belongs in the default. Every user of eden platform will have its own domain.
So tld element is DETACHED DEFAULT. It must be defined by the user, and can only ever be defined once. If it is not defined, or defined multiple times it is compiler error.

Code: [Select]

DEFAULTS {
  // defines default for table 'server' and column 'tld'.
  // you cannot define defaults for non existing tables
  // and column must be marked as detached default
  server.tld epl-infra.net,
}

// now we can define data with detached default
DATA STRUCT server {
  hostname: server-a
}

DATA tld {
  epl-infra.net;
}
Full example here https://gist.github.com/cultleader777/3823ccef5c22b4b086c2468ab9e2e89c

And these are the main features I needed to add to EdenDB so far while working on eden platform compiler.

See you later bois!