dtServices

A dtService is a Kubernetes CRD that allows creation of Wireguard networks between pods on different nodes of a federation. Each network is identified by the name of the dtService object. Creating a dtService with a name not already associated with a network will create a new one while using the name of an existing network will cause the associated pod to join the existing network.

Example

The yaml for adding a dtService to Kubernetes looks like:

apiVersion: sid.sightlineinnovation.com/v1
kind: dtService
metadata:
  name: myservice
spec:
  domainSelector: mydomain
  podSelector: my-pod

This will create a new network associated with the name myservice and add a network interface for this network to the pod my-pod. Due to the nature of kubernetes networking, all containers in the pod will have access to this new network interface. The domainSelector field here indicates that the dtService is associated with the federation defined by the dtDomain mydomain and only other nodes in this federation may add new pods to the network.

For more information on creation of a federation see here.

You can view the updated dtService using kubectl.

$ kubectl describe dtservices

Name:         myservice
Namespace:    dtaas-node0-dtaas
Labels:       <none>
Annotations:  <none>
API Version:  sid.sightlineinnovation.com/v1
Kind:         dtService
Metadata:
  Creation Timestamp:  2021-06-21T17:09:20Z
  Generation:          4
  Resource Version:  9361685
  Self Link:         /apis/sid.sightlineinnovation.com/v1/namespaces/dtaas-node0-dtaas/dtservices/testdtservice
  UID:               b4c5e0a5-dbff-49e2-bee6-eacbfb5ce3ce
Peers:
  100.96.0.1:
    Labels:
      sid.sightline.dtservice.74dbd2ec528e4db3a496a893f85cd0d2:  server
    Public Key:                                                  d7Q5KQuLcr4VTxJcwsZmtDoq7fEpyYTuR1x0BWiaEXw=
Spec:
  Domain Selector:  mydomain
  Pod Selector:     my-pod
Status:             Connected
Events:             <none>

At this point there are two changes to note, the addition of the Status and Peers. The Status above indicates that we’re connected to the network. This is also where it will be indicated if we failed to create or join the network. The Peers are a mapping of IPv4 addresses to metadata about the pod at that address. The metadata consists of the Wireguard public key for that peer and the labels of the pod at that address. The peers are managed by DTaaS and should not be modified directly.

Once we have a network created we may add more pods on different nodes to the network. The yaml for doing doing so looks like:

apiVersion: sid.sightlineinnovation.com/v1
kind: dtService
metadata:
  name: myservice
spec:
  domainSelector: mydomain
  podSelector: service-pod

Notice that the above looks almost identical to the yaml used for creating the network. The name must be the same but the domainSelector and podSelector may be different depending on how the dtDomain was named and what pod we wish to attach to the network.

If we look at the dtService in Kubernetes at this point we will get:

$ kubectl describe dtservices

Name:         myservice
Namespace:    dtaas-node1-dtaas
Labels:       <none>
Annotations:  <none>
API Version:  sid.sightlineinnovation.com/v1
Kind:         dtService
Metadata:
  Creation Timestamp:  2021-06-21T17:28:34Z
  Generation:          5
  Resource Version:  9364231
  Self Link:         /apis/sid.sightlineinnovation.com/v1/namespaces/dtaas-node1-dtaas/dtservices/testdtservice
  UID:               839bbb8d-49f6-4c7b-9c01-585ea5b8294a
Peers:
  100.96.0.1:
    Labels:
      sid.sightline.dtservice.74dbd2ec528e4db3a496a893f85cd0d2:  server
    Public Key:                                                  d7Q5KQuLcr4VTxJcwsZmtDoq7fEpyYTuR1x0BWiaEXw=
  100.96.0.2:
    Labels:
      sid.sightline.dtservice.5687b751a5c941679864d072bfd30e51:  client
    Public Key:                                                  0/0DnGBZH2wc81p25+Kygj9KxLZu2vjWsX8kte/GUR4=
Spec:
  Domain Selector:  mydomain
  Pod Selector:     service-pod
  Server:
    Endpoint:    10.4.0.16:51830
    Peer:        100.96.0.0/24
    Public Key:  d7Q5KQuLcr4VTxJcwsZmtDoq7fEpyYTuR1x0BWiaEXw=
Status:          Connected
Events:          <none>

This looks similar to the result after creating the network but we now have a new field called Server as part of the spec. This field contains the connection information needed for adding the connected pod as a client on the Wireguard network. This information is generated automatically by DTaaS and as such shouldn’t be set or modified manually.

Ethereum Example

One potential use-case for a dtService is to run a private ethereum blockchain within a federation. This is accomplished by deploying an ethereum node on each DTaaS node in the federation.

The first step to deploying the private ethereum blockchain is ot deploy the initial node. This will act as a boot node to facilitate peering. This node can be deployed however you want, only requiring that it be in the same kubernetes namespace as DTaaS.

Once we have our initial ethereum node, we need to create the dtService for it. This is as simple as creating a dtService object in kubernetes that looks like:

apiVersion: sid.sightlineinnovation.com/v1
kind: dtService
metadata:
  name: ethereumservice
spec:
  domainSelector: mydomain
  podSelector: ethereum

The above assumes that our dtDomain has been named mydomain and our ethereum node is running in a pod called ethereum. After having been applied by DTaaS, the dtService in kubernetes should look something like:

$ kubectl describe dtservices

Name:         ethereumservice
Namespace:    dtaas-node0-dtaas
Labels:       <none>
Annotations:  <none>
API Version:  sid.sightlineinnovation.com/v1
Kind:         dtService
Metadata:
  Creation Timestamp:  2021-06-21T17:09:20Z
  Generation:          4
  Resource Version:  9361685
  Self Link:         /apis/sid.sightlineinnovation.com/v1/namespaces/dtaas-node0-dtaas/dtservices/ethereumservice
  UID:               b4c5e0a5-dbff-49e2-bee6-eacbfb5ce3ce
Peers:
  100.96.0.1:
    Labels:
      sid.sightline.dtservice.74dbd2ec528e4db3a496a893f85cd0d2:  server
    Public Key:                                                  d7Q5KQuLcr4VTxJcwsZmtDoq7fEpyYTuR1x0BWiaEXw=
Spec:
  Domain Selector:  mydomain
  Pod Selector:     ethereum
Status:             Connected
Events:             <none>

Of interest to us in the above is the peers. We can see that our only peer has been assigned an IP of 100.96.0.1. This is how other nodes will connect to us.

the next thing that’s needed is the enode value of the ethereum node. This should look something like:

"enode://e0fa4355860d1f42ab6c3a3e918b0191a7a58712c4bdef74657daca282205197db2609082d9cdd023e801676316422bc6ed0cc3df0dfa10359751b3b521d6503@10.1.111.77:30303"

What we need to do is replace the IP address at the end with the address from our dtService, giving us:

"enode://e0fa4355860d1f42ab6c3a3e918b0191a7a58712c4bdef74657daca282205197db2609082d9cdd023e801676316422bc6ed0cc3df0dfa10359751b3b521d6503@100.96.0.1:30303"

This updated enode string can then be shared with anyone else in the federation deploying an ethereum node along with the name of the dtService. They can deploy their node normally, either adding the enode value supplied ot them as a boot node or explicitly adding it as a peer. They will also need to create a dtService themselves. It should be structures similarly to the boot node yaml and have the same name but the pod name and dtDomain name may be different.

Note

When adding a new ethereum node, the corresponding dtService may be created first. This is advisable in most cases as it ensures that the pod attempts to connect to the dtService network on startup, reducing the likelihood of failing to connect to the boot node initially and having to wait for it to reattempt connection.