dtServices¶
A dtService
is a Kubernetes CRD that allows creation of Wireguard networks
between pods on different nodes of a federation. Each network is identified by
the name of the dtService object. Creating a dtService
with a name not
already associated with a network will create a new one while using the name of
an existing network will cause the associated pod to join the existing network.
Example¶
The yaml for adding a dtService
to Kubernetes looks like:
apiVersion: sid.sightlineinnovation.com/v1
kind: dtService
metadata:
name: myservice
spec:
domainSelector: mydomain
podSelector: my-pod
This will create a new network associated with the name myservice
and add
a network interface for this network to the pod my-pod
. Due to the
nature of kubernetes networking, all containers in the pod will have access to
this new network interface. The domainSelector
field here indicates that
the dtService
is associated with the federation defined by the dtDomain
mydomain
and only other nodes in this federation may add new pods to the
network.
For more information on creation of a federation see here.
You can view the updated dtService
using kubectl
.
$ kubectl describe dtservices
Name: myservice
Namespace: dtaas-node0-dtaas
Labels: <none>
Annotations: <none>
API Version: sid.sightlineinnovation.com/v1
Kind: dtService
Metadata:
Creation Timestamp: 2021-06-21T17:09:20Z
Generation: 4
Resource Version: 9361685
Self Link: /apis/sid.sightlineinnovation.com/v1/namespaces/dtaas-node0-dtaas/dtservices/testdtservice
UID: b4c5e0a5-dbff-49e2-bee6-eacbfb5ce3ce
Peers:
100.96.0.1:
Labels:
sid.sightline.dtservice.74dbd2ec528e4db3a496a893f85cd0d2: server
Public Key: d7Q5KQuLcr4VTxJcwsZmtDoq7fEpyYTuR1x0BWiaEXw=
Spec:
Domain Selector: mydomain
Pod Selector: my-pod
Status: Connected
Events: <none>
At this point there are two changes to note, the addition of the Status
and
Peers
. The Status
above indicates that we’re connected to the network.
This is also where it will be indicated if we failed to create or join the
network. The Peers
are a mapping of IPv4 addresses to metadata about the
pod at that address. The metadata consists of the Wireguard public key for that
peer and the labels of the pod at that address. The peers are managed by DTaaS
and should not be modified directly.
Once we have a network created we may add more pods on different nodes to the network. The yaml for doing doing so looks like:
apiVersion: sid.sightlineinnovation.com/v1
kind: dtService
metadata:
name: myservice
spec:
domainSelector: mydomain
podSelector: service-pod
Notice that the above looks almost identical to the yaml used for creating the
network. The name
must be the same but the domainSelector
and
podSelector
may be different depending on how the dtDomain
was named
and what pod we wish to attach to the network.
If we look at the dtService
in Kubernetes at this point we will get:
$ kubectl describe dtservices
Name: myservice
Namespace: dtaas-node1-dtaas
Labels: <none>
Annotations: <none>
API Version: sid.sightlineinnovation.com/v1
Kind: dtService
Metadata:
Creation Timestamp: 2021-06-21T17:28:34Z
Generation: 5
Resource Version: 9364231
Self Link: /apis/sid.sightlineinnovation.com/v1/namespaces/dtaas-node1-dtaas/dtservices/testdtservice
UID: 839bbb8d-49f6-4c7b-9c01-585ea5b8294a
Peers:
100.96.0.1:
Labels:
sid.sightline.dtservice.74dbd2ec528e4db3a496a893f85cd0d2: server
Public Key: d7Q5KQuLcr4VTxJcwsZmtDoq7fEpyYTuR1x0BWiaEXw=
100.96.0.2:
Labels:
sid.sightline.dtservice.5687b751a5c941679864d072bfd30e51: client
Public Key: 0/0DnGBZH2wc81p25+Kygj9KxLZu2vjWsX8kte/GUR4=
Spec:
Domain Selector: mydomain
Pod Selector: service-pod
Server:
Endpoint: 10.4.0.16:51830
Peer: 100.96.0.0/24
Public Key: d7Q5KQuLcr4VTxJcwsZmtDoq7fEpyYTuR1x0BWiaEXw=
Status: Connected
Events: <none>
This looks similar to the result after creating the network but we now have a
new field called Server
as part of the spec
. This field contains the
connection information needed for adding the connected pod as a client on the
Wireguard network. This information is generated automatically by DTaaS and as
such shouldn’t be set or modified manually.
Ethereum Example¶
One potential use-case for a dtService
is to run a private ethereum blockchain
within a federation. This is accomplished by deploying an ethereum node on each
DTaaS node in the federation.
The first step to deploying the private ethereum blockchain is ot deploy the initial node. This will act as a boot node to facilitate peering. This node can be deployed however you want, only requiring that it be in the same kubernetes namespace as DTaaS.
Once we have our initial ethereum node, we need to create the dtService for it.
This is as simple as creating a dtService
object in kubernetes that looks like:
apiVersion: sid.sightlineinnovation.com/v1
kind: dtService
metadata:
name: ethereumservice
spec:
domainSelector: mydomain
podSelector: ethereum
The above assumes that our dtDomain
has been named mydomain
and our
ethereum node is running in a pod called ethereum
. After having been
applied by DTaaS, the dtService
in kubernetes should look something like:
$ kubectl describe dtservices
Name: ethereumservice
Namespace: dtaas-node0-dtaas
Labels: <none>
Annotations: <none>
API Version: sid.sightlineinnovation.com/v1
Kind: dtService
Metadata:
Creation Timestamp: 2021-06-21T17:09:20Z
Generation: 4
Resource Version: 9361685
Self Link: /apis/sid.sightlineinnovation.com/v1/namespaces/dtaas-node0-dtaas/dtservices/ethereumservice
UID: b4c5e0a5-dbff-49e2-bee6-eacbfb5ce3ce
Peers:
100.96.0.1:
Labels:
sid.sightline.dtservice.74dbd2ec528e4db3a496a893f85cd0d2: server
Public Key: d7Q5KQuLcr4VTxJcwsZmtDoq7fEpyYTuR1x0BWiaEXw=
Spec:
Domain Selector: mydomain
Pod Selector: ethereum
Status: Connected
Events: <none>
Of interest to us in the above is the peers. We can see that our only peer has
been assigned an IP of 100.96.0.1
. This is how other nodes will connect to
us.
the next thing that’s needed is the enode value of the ethereum node. This should look something like:
"enode://e0fa4355860d1f42ab6c3a3e918b0191a7a58712c4bdef74657daca282205197db2609082d9cdd023e801676316422bc6ed0cc3df0dfa10359751b3b521d6503@10.1.111.77:30303"
What we need to do is replace the IP address at the end with the address from
our dtService
, giving us:
"enode://e0fa4355860d1f42ab6c3a3e918b0191a7a58712c4bdef74657daca282205197db2609082d9cdd023e801676316422bc6ed0cc3df0dfa10359751b3b521d6503@100.96.0.1:30303"
This updated enode string can then be shared with anyone else in the federation
deploying an ethereum node along with the name of the dtService
. They can
deploy their node normally, either adding the enode value supplied ot them as a
boot node or explicitly adding it as a peer. They will also need to create a
dtService
themselves. It should be structures similarly to the
boot node yaml and have the same name but the pod name and dtDomain
name
may be different.
Note
When adding a new ethereum node, the corresponding dtService
may be
created first. This is advisable in most cases as it ensures that the pod
attempts to connect to the dtService
network on startup, reducing the
likelihood of failing to connect to the boot node initially and having to
wait for it to reattempt connection.