It so happened that I am an administrator of computer systems and networks by profession (in short: a system administrator), and I had a chance to tell prof. activities of a wide variety of systems, including those that require [extra]high security measures. It also happened that some time ago I found it interesting , and not only used it, but also launched several micro-services in order to learn how to independently work with the bitcoin network (aka p2p after all) from the point of view of a developer (of course, I dev, so, passed by). But I'm not talking about development, I'm talking about a safe and efficient environment for applications.
Financial Technology (fintech) go next to information security (infosec) and the first without the second can work, but not for long. That's why I want to share my experience and toolset that I use, which includes like fintechAnd infosec, and at the same time, and can also be used in a wider or completely different purpose. In this article, I will talk not so much about bitcoin, but about the infrastructure model for the development and operation of financial (and not only) services - in a word, those services where “B” matters. This applies both to the bitcoin exchange and to the most typical corporate zoo of services of a small company with no connection to bitcoin.
I want to note that I am a supporter of the principles "keep it stupid simple" и "less is more", therefore, both the article and what is described in it will have the properties about which these principles are.
Imaginary scenario: Let's look at everything using the example of a bitcoin exchanger. We decided to launch the exchange of rubles, dollars, euros for bitcoins and vice versa, and we already have a working solution, but for other digital money like qiwi and webmoney, i.e. we have closed all legal issues, we have a ready-made application that acts as a payment gateway for rubles, dollars and euros and other payment systems. It is tied to our bank accounts and has some kind of API for our end applications. We also have a web application that acts as an exchanger for users, well, like a typical qiwi or webmoney cabinet - create an account, add a card, and so on. It communicates with our gateway application, albeit via the REST API in the local area. And so we decided to connect bitcoins and at the same time upgrade the infrastructure, because. Initially, everything was in a hurry raised on virtualboxes in the office under the table ... the site began to be used, and we began to worry about uptime and performance.
So, let's start with the main one - choosing a server. Because the business in our example is small and we trust the hosting provider (OVH) we will choose in which you cannot install the system from the original .iso image, but it doesn’t matter, the IT security department will definitely analyze the installed image. And when we grow up, we generally rent our closet under lock and key with limited physical access, and maybe we will build our own DC. In any case, it is worth remembering that when renting hardware and installing ready-made images, there is a chance that you will have a "trojan from the hoster" hanging in your system, which in most cases is not intended to spy on you, but to offer more convenient management tools server.
Server installation
Everything is simple here. We choose the iron that suits our needs. Then select the FreeBSD image. Well, or we connect (in the case of another hoster and our own hardware) via IPMI or with a monitor and feed the .iso FreeBSD image into the boot. For orchestral setup I use и . The only thing, in our case with kimsufi, we chose custom installation in order for two disks in the mirror to have “open” only the boot and / home partitions, the rest of the disk space will be encrypted, but more on that later.
The system is installed in a standard way, I will not dwell on this, I will only note that before starting operation, you should pay attention to hardening options that offer bsdinstaller at the end of the installation (if you install the system yourself):
There is on this topic, I will briefly repeat it here.
It is also possible to enable the above parameters on an already installed system. To do this, you need to edit the bootloader file and enable kernel options. *ee is the BSD editor
# ee /etc/rc.conf
...
#sec hard
clear_tmp_enable="YES"
syslogd_flags="-ss"
sendmail_enable="NONE"
# ee /etc/sysctl.conf
...
#sec hard
security.bsd.see_other_uids=0
security.bsd.see_other_gids=0
security.bsd.unprivileged_read_msgbuf=0
security.bsd.unprivileged_proc_debug=0
kern.randompid=$(jot -r 1 9999)
security.bsd.stack_guard_page=1It is also worth making sure that you have the latest version of the system installed, and . In our case, for example, an upgrade to the latest version is required, because. pre-installation images are six months or a year behind. Well, there we change the SSH port to a different one from the default, add authentication by keys and disable by password.
Then we set up aide, monitoring the status of system configuration files. You can read more elaborately .
pkg install aide
and edit our crontab
crontab -e
06 01 * * 0-6 /root/chkaide.sh
#! /bin/sh
#chkaide.sh
MYDATE=`date +%Y-%m-%d`
MYFILENAME="Aide-"$MYDATE.txt
/bin/echo "Aide check !! `date`" > /tmp/$MYFILENAME
/usr/local/bin/aide --check > /tmp/myAide.txt
/bin/cat /tmp/myAide.txt|/usr/bin/grep -v failed >> /tmp/$MYFILENAME
/bin/echo "**************************************" >> /tmp/$MYFILENAME
/usr/bin/tail -20 /tmp/myAide.txt >> /tmp/$MYFILENAME
/bin/echo "****************DONE******************" >> /tmp/$MYFILENAMETurn on
sysrc auditd_enable=YES
# service auditd start
How to administer this business is perfectly described in .
Now we reboot and proceed to the software on the server. Each server is a hypervisor for containers or full virtual machines. Therefore, it is important that the processor supports VT-x and EPT if we plan to use full virtualization.
As a management of containers and virtual machines, I use from , I wish him more health and blessings for this wonderful utility!
Containers? Docker again or what?
And no. is a great tool for containerization, but the mentioned cbsd to orchestrate these containers, whose name is cells.
A cage is an extremely effective solution for building infrastructure for a variety of purposes, where, as a result, complete isolation of individual services or processes is required. It is essentially a clone of the host system, but does not require full hardware virtualization. And thanks to this, resources are not spent on the "guest OS", but only on the work performed. When cells are used for internal needs, this is a very convenient solution for optimal use of the resource - a bunch of cells on one iron server can each individually use the entire server resource if necessary. Considering that usually different subservices need additional. resources at different times, you can extract maximum performance from one server if you plan and unbalance cells between servers correctly. If necessary, cells can also set limits on the resource used.
What about full virtualization?
As far as I know, cbsd supports work bhyve and XEN hypervisors. I have never used the second one, but the first one is relatively young . We will look at an example of using bhyve in the example below.
Installing and configuring the host environment
We use FS . It is an extremely powerful tool for managing server space. Thanks to ZFS, you can directly build arrays of various configurations from disks, dynamically expand space “hot”, change dead disks, manage snapshots, and much, much more that can be described in a whole series of articles. Let's return to our server and its disks. At the beginning of the installation on the disks, we left free space for encrypted partitions. Why is that? This is to ensure that the system rises automatically and listens via SSH.
gpart add -t freebsd-zfs /dev/ada0
/dev/ada0p4 added!
add a disk partition to the remaining space
geli init /dev/ada0p4
we drive in our encryption password
geli attach /dev/ada0p4
again we enter the password and we have a device /dev/ada0p4.eli - this is our encrypted space. Then we repeat the same for /dev/ada1 and the rest of the disks in the array. And we create a new one .
zpool create vms mirror /dev/ada0p4.eli /dev/ada1p4.eli /dev/ada3p4.eli - well, we have a minimum combat set ready. A mirror array of disks in case one of the three fails.
We create a dataset on a new “pool”
zfs create vms/jails
pkg install cbsd - launched the team, and set the management for our cells.
After cbsd installed, it needs to be initialized:
# env workdir="/vms/jails" /usr/local/cbsd/sudoexec/initenv
well, we answer a bunch of questions, mostly with default answers.
*If you are using encryption, it is important that the daemon cbsdd did not start automatically until you decrypt the disks manually or automatically (in our example, this is done by zabbix)
**Also, I don't use NAT from cbsd, but I configure it myself in pf.
# sysrc pf_enable=YES
# ee /etc/pf.conf
IF_PUBLIC="em0"
IP_PUBLIC="1.23.34.56"
JAIL_IP_POOL="192.168.0.0/24"
#WHITE_CL="{ 127.0.0.1 }"
icmp_types="echoreq"
set limit { states 20000, frags 20000, src-nodes 20000 }
set skip on lo0
scrub in all
#NAT for jails
nat pass on $IF_PUBLIC from $JAIL_IP_POOL to any -> $IP_PUBLIC
## Bitcoin network port forward
IP_JAIL="192.168.0.1"
PORT_JAIL="{8333}"
rdr pass on $IF_PUBLIC proto tcp from any to $IP_PUBLIC port $PORT_JAIL -> $IP_JAIL
# service pf start
# pfctl -f /etc/pf.conf
Setting up firewall policies is also a separate topic, so I will not delve into setting up the BLOCK ALL policy and setting up whitelists, this can be done by reading or any of the huge number of articles available on Google.
Alright… we have cbsd installed, time to build our first workhorse, the caged bitcoin daemon!
cbsd jconstruct-tui
Here we see the cell creation dialog. After all the values were set, we create!
When creating the first cell, you should choose what to use as the basis for the cells. I select a distribution from the FreeBSD repository with the command repo. This choice is made only when creating the first cage of a particular version (you can host cages of any version that is older than the host version).
After everything is installed, we launch the cage!
# cbsd jstart bitcoind
But we need to install software in the cage.
# jls
JID IP Address Hostname Path
1 192.168.0.1 bitcoind.space.com /zroot/jails/jails/bitcoindjexec bitcoind to get into the cell console
and already inside the cage we install the software with its dependencies (our host system remains clean)
bitcoind:/@[15:25] # pkg install bitcoin-daemon bitcoin-utils
bitcoind:/@[15:30] # sysrc bitcoind_enable=YES
bitcoind:/@[15:30] # service bitcoind start
There is Bitcoin in the cage, but we need anonymity, because we want to connect to some cages via the TOR network. In general, we plan to rotate most of the cells with suspicious software only through a proxy. Thanks to pf you can disable NAT for a specific range of IP addresses in the local network, and allow NAT only for our TOR node. Thus, even if malware enters the cell, it most likely will not contact the outside world, and if it does, it will not reveal the IP of our server. therefore, we create another cell to "forward" services as a ".onion" service and as a proxy for accessing the Internet to individual cells.
# cbsd jsconstruct-tui
# cbsd jstart tor
# jexec tor
tor:/@[15:38] # pkg install tor
tor:/@[15:38] # sysrc tor_enable=YES
tor:/@[15:38] # ee /usr/local/etc/tor/torrc
We set to listen on the local address (available for all cells)
SOCKSPort 192.168.0.2:9050
What else do we lack for complete happiness. Yes, we need a service for our web, maybe more than one. Let's start nginx, which will act as a reverse-proxy and take care of renewing Let's Encrypt certificates
# cbsd jsconstruct-tui
# cbsd jstart nginx-rev
# jexec nginx-rev
nginx-rev:/@[15:47] # pkg install nginx py36-certbot
And here we put 150 MB of dependencies in a cage. And the host is still clean.
Let's get back to setting up nginx later, we need to raise two more cells for our payment gateway on nodejs and rust and a web application, which for some reason is on Apache and PHP, and the latter also needs a MySQL database.
# cbsd jsconstruct-tui
# cbsd jstart paygw
# jexec paygw
paygw:/@[15:55] # pkg install git node npm
paygw:/@[15:55] # curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
... and another 380 MB of packages isolated
then we download our application with git and run it.
# cbsd jsconstruct-tui
# cbsd jstart webapp
# jexec webapp
webapp:/@[16:02] # pkg install mariadb104-server apache24 php74 mod_php74 php74-pdo_mysql
450 MB packages. in a cage.
here we give the developer access via SSH directly into the cage, they will do everything there themselves:
webapp:/@[16:02] # ee /etc/ssh/sshd_config
Port 2267 - change the SSH port of the cell to any arbitrary
webapp:/@[16:02] # sysrc sshd_enable=YES
webapp:/@[16:02] # service sshd start
Well, the service is running, it remains to add the rule to pf firewall
Let's see what IPs we have for cells and how our “locale” looks in general
# jls
JID IP Address Hostname Path
1 192.168.0.1 bitcoind.space.com /zroot/jails/jails/bitcoind
2 192.168.0.2 tor.space.com /zroot/jails/jails/tor
3 192.168.0.3 nginx-rev.space.com /zroot/jails/jails/nginx-rev
4 192.168.0.4 paygw.space.com /zroot/jails/jails/paygw
5 192.168.0.5 webapp.my.domain /zroot/jails/jails/webappand add a rule
# ee /etc/pf.conf
## SSH for web-Devs
IP_JAIL="192.168.0.5"
PORT_JAIL="{ 2267 }"
rdr pass on $IF_PUBLIC proto tcp from any to $IP_PUBLIC port $PORT_JAIL -> $IP_JAIL
well, since we are here, we will also add a rule on reverse-proxy:
## web-ports for nginx-rev
IP_JAIL="192.168.0.3"
PORT_JAIL="{ 80, 443 }"
rdr pass on $IF_PUBLIC proto tcp from any to $IP_PUBLIC port $PORT_JAIL -> $IP_JAIL# pfctl -f /etc/pf.conf
Well, now a little bit about bitcoins.
What we have is we have a web application that is accessible from the outside, and it communicates locally with our payment gateway. Now we need to prepare a working environment for interacting with the bitcoin network itself - node bitcoind it's just a daemon that keeps a local copy of the blockchain up to date. This daemon has RPC and wallet functionality, but there are more convenient "wrappers" for application development. To begin with, we decided to put electrum is a CLI wallet. we will use it as a “cold storage” for our bitcoins - in general, those bitcoins that will need to be stored “outside” the system accessible to users and generally away from everyone. It also has a GUI, so we are going to use the same wallet on our website.
laptops. For now, we will use electrum with public servers, and later in another cell we will raise so as not to depend on anyone at all.
# cbsd jsconstruct-tui
# cbsd jstart electrum
# jexec electrum
electrum:/@[8:45] # pkg install py36-electrum
another 700 MB of software in our cage
electrum:/@[8:53] # adduser
Username: wallet
Full name:
Uid (Leave empty for default):
Login group [wallet]:
Login group is wallet. Invite wallet into other groups? []:
Login class [default]:
Shell (sh csh tcsh nologin) [sh]: tcsh
Home directory [/home/wallet]:
Home directory permissions (Leave empty for default):
Use password-based authentication? [yes]: no
Lock out the account after creation? [no]:
Username : wallet
Password : <disabled>
Full Name :
Uid : 1001
Class :
Groups : wallet
Home : /home/wallet
Home Mode :
Shell : /bin/tcsh
Locked : no
OK? (yes/no): yes
adduser: INFO: Successfully added (wallet) to the user database.
Add another user? (yes/no): no
Goodbye!
electrum:/@[8:53] # su walletelectrum:/@[8:53] # su wallet
wallet@electrum:/ % electrum-3.6 create
{
"msg": "Please keep your seed in a safe place; if you lose it, you will not be able to restore your wallet.",
"path": "/usr/home/wallet/.electrum/wallets/default_wallet",
"seed": "jealous win pig material ribbon young punch visual okay cactus random bird"
}Now we have created a wallet.
wallet@electrum:/ % electrum-3.6 listaddresses
[
"18WEhbjvMLGRMfwudzUrUd25U5C7uZYkzE",
"14XHSejhxsZNDRtk4eFbqAX3L8rftzwQQU",
"1KQXaN8RXiCN1ne9iYngUWAr6KJ6d4pPas",
...
"1KeVcAwEYhk29qEyAfPwcBgF5mMMoy4qjw",
"18VaUuSeBr6T2GwpSHYF3XyNgLyLCt1SWk"
]wallet@electrum:/ % electrum-3.6 help
To our on-chain From now on, only a limited circle of people will be able to connect to the wallet. In order not to open access from the outside to this cell, SSH connections will occur through TOR (such a decentralized version of VPN). We start SSH in the cage, but do not touch our pf.conf on the host.
electrum:/@[9:00] # sysrc sshd_enable=YES
electrum:/@[9:00] # service sshd start
Now let's turn off the Internet connection in the cage with the wallet. Let's give it an IP address from another subnet space that is not NATized. Let's change first /etc/pf.conf on the host
# ee /etc/pf.conf
JAIL_IP_POOL="192.168.0.0/24" change to JAIL_IP_POOL="192.168.0.0/25", thus all addresses 192.168.0.126-255 will not have direct access to the Internet. A kind of software "air-gap" network. And the NAT rule remains as it was
nat pass on $IF_PUBLIC from $JAIL_IP_POOL to any -> $IP_PUBLIC
Overloading the rules
# pfctl -f /etc/pf.conf
Now we take on our cage
# cbsd jconfig jname=electrum
jset mode=quiet jname=electrum ip4_addr="192.168.0.200"
Remove old IP: /sbin/ifconfig em0 inet 192.168.0.6 -alias
Setup new IP: /sbin/ifconfig em0 inet 192.168.0.200 alias
ip4_addr: 192.168.0.200Hmm, but now the system itself will stop working for us. However, we can specify a system proxy. But there is one thing, on TOR it is a SOCKS5 proxy, and for convenience, we would also have an HTTP proxy.
# cbsd jsconstruct-tui
# cbsd jstart polipo
# jexec polipo
polipo:/@[9:28] # pkg install polipo
polipo:/@[9:28] # ee /usr/local/etc/polipo/config
socksParentProxy = "192.168.0.2:9050"
socksProxyType = socks5polipo:/@[9:42] # sysrc polipo_enable=YES
polipo:/@[9:43] # service polipo start
Well, now we have two proxy servers in our system, and both output via TOR: socks5://192.168.0.2:9050 and
Now we can set up our wallet environment
# jexec electrum
electrum:/@[9:45] # su wallet
wallet@electrum:/ % ee ~/.cshrc
#in the end of file proxy config
setenv http_proxy http://192.168.0.6:8123
setenv https_proxy http://192.168.0.6:8123Well, now the shell will work from under the proxy. If we want to install packages, then we should add to /usr/local/etc/pkg.conf from under the root of the cell
pkg_env: {
http_proxy: "http://my_proxy_ip:8123",
}Well, now it's time to add the TOR hidden service as the address of our SSH service in the wallet cage.
# jexec tor
tor:/@[9:59] # ee /usr/local/etc/tor/torrc
HiddenServiceDir /var/db/tor/electrum/
HiddenServicePort 22 192.168.0.200:22tor:/@[10:01] # mkdir /var/db/tor/electrum
tor:/@[10:01] # chown -R _tor:_tor /var/db/tor/electrum
tor:/@[10:01] # chmod 700 /var/db/tor/electrum
tor:/@[10:03] # service tor restart
tor:/@[10:04] # cat /var/db/tor/electrum/hostname
mdjus4gmduhofwcso57b3zl3ufoitguh2knitjco5cmgrokpreuxumad.onionHere is our connection address. Let's check from the local machine. But first we need to add our SSH key:
wallet@electrum:/ % mkdir ~/.ssh
wallet@electrum:/ % ee ~/.ssh/authorized_keys
ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAG9Fk2Lqi4GQ8EXZrsH3EgSrVIQPQaAlS38MmJLBabihv9KHIDGXH7r018hxqLNNGbaJWO/wrWk7sG4T0yLHAbdQAFsMYof9kjoyuG56z0XZ8qaD/X/AjrhLMsIoBbUNj0AzxjKNlPJL4NbHsFwbmxGulKS0PdAD5oLcTQi/VnNdU7iFw== user@localWell, from a Linux client machine
user@local ~$ nano ~/.ssh/config
#remote electrum wallet
Host remotebtc
User wallet
Port 22
Hostname mdjus4gmduhofwcso57b3zl3ufoitguh2knitjco5cmgrokpreuxumad.onion
ProxyCommand /bin/ncat --proxy localhost:9050 --proxy-type socks5 %h %p
Connecting (In order for this to work, you need a local TOR daemon that listens on 9050)
user@local ~$ ssh remotebtc
The authenticity of host 'mdjus4gmduhofwcso57b3zl3ufoitguh2knitjco5cmgrokpreuxumad.onion (<no hostip for proxy command>)' can't be established.
ECDSA key fingerprint is SHA256:iW8FKjhVF4yyOZB1z4sBkzyvCM+evQ9cCL/EuWm0Du4.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'mdjus4gmduhofwcso57b3zl3ufoitguh2knitjco5cmgrokpreuxumad.onion' (ECDSA) to the list of known hosts.
FreeBSD 12.1-RELEASE-p1 GENERIC
To save disk space in your home directory, compress files you rarely
use with "gzip filename".
-- Dru <[email protected]>
wallet@electrum:~ % logout
Success!
To work with instant and micro-payments, we also need a node , in fact, this will be our main working tool with bitcoin. *, which we are going to use as a daemon, is , which is a full-fledged HTTP (REST) interface and allows you to work with both off-chain transactions and on-chain transactions. c-lightning needed to function bitcoind but yes.
*there are different implementations on different PLs of the Lightning Network protocol. Of the ones we tested, c-lightning (written in C) seemed to be the most stable and resource efficient.
# cbsd jsconstruct-tui
# cbsd jstart cln
# jexec cln
lightning:/@[10:23] # adduser
Username: lightning
...lightning:/@[10:24] # pkg install git
lightning:/@[10:23] # su lightning
cd ~ && git clone https://github.com/ElementsProject/lightning
lightning@lightning:~ % exit
lightning:/@[10:30] # cd /home/lightning/lightning/
lightning:/home/lightning/lightning@[10:31] # pkg install autoconf automake gettext git gmp gmake libtool python python3 sqlite3 libsodium py36-mako bash bitcoin-utils
lightning:/home/lightning/lightning@[10:34] # ./configure && gmake && gmake install
While compiling and installing everything you need, let's create an RPC user for lightningd в bitcoind
# jexec bitcoind
bitcoind:/@[10:36] # ee /usr/local/etc/bitcoin.conf
rpcbind=192.168.0.1
rpcuser=test
rpcpassword=test
#allow only c-lightning
rpcallowip=192.168.0.7/32bitcoind:/@[10:39] # service bitcoind restart
My chaotic switching between cells turns out to be not so chaotic after all if I check the utility tmux, which allows you to create many sub-sessions of terminals within one session. Analog: screen
Ok, we do not want to reveal the real IP of our node, and we want to conduct all financial transactions through the TOR. Therefore, one more .onion is needed.
# jexec tor
tor:/@[9:59] # ee /usr/local/etc/tor/torrc
HiddenServiceDir /var/db/tor/cln/
HiddenServicePort 9735 192.168.0.7:9735tor:/@[10:01] # mkdir /var/db/tor/cln
tor:/@[10:01] # chown -R _tor:_tor /var/db/tor/cln
tor:/@[10:01] # chmod 700 /var/db/tor/cln
tor:/@[10:03] # service tor restart
tor:/@[10:04] # cat /var/db/tor/cln/hostname
en5wbkavnytti334jc5uzaudkansypfs6aguv6kech4hbzpcz2ove3yd.onionnow let's create a config for c-lightning
lightning:/home/lightning/lightning@[10:31] # su lightning
lightning@lightning:~ % mkdir .lightning
lightning@lightning:~ % ee .lightning/config
alias=My-LN-Node
bind-addr=192.168.0.7:9735
rgb=ff0000
announce-addr=en5wbkavnytti334jc5uzaudkansypfs6aguv6kech4hbzpcz2ove3yd.onion:9735
network=bitcoin
log-level=info
fee-base=0
fee-per-satoshi=1
proxy=192.168.0.2:9050
log-file=/home/lightning/.lightning/c-lightning.log
min-capacity-sat=200000
# sparko plugin
# https://github.com/fiatjaf/lightningd-gjson-rpc/tree/master/cmd/sparko
sparko-host=192.168.0.7
sparko-port=9737
sparko-tls-path=sparko-tls
#sparko-login=mywalletusername:mywalletpassword
#sparko-keys=masterkey;secretread:+listchannels,+listnodes;secretwrite:+invoice,+listinvoices,+delinvoice,+decodepay,+waitpay,+waitinvoice
sparko-keys=masterkey;secretread:+listchannels,+listnodes;ultrawrite:+invoice,+listinvoices,+delinvoice,+decodepay,+waitpay,+waitinvoice
# for the example above the initialization logs (mixed with lightningd logs) should print something likelightning@lightning:~ % mkdir .lightning/plugins
lightning@lightning:~ % cd .lightning/plugins/
lightning@lightning:~/.lightning/plugins:% fetch https://github.com/fiatjaf/sparko/releases/download/v0.2.1/sparko_full_freebsd_amd64
lightning@lightning:~/.lightning/plugins % mkdir ~/.lightning/sparko-tls
lightning@lightning:~/.lightning/sparko-tls % cd ~/.lightning/sparko-tls
lightning@lightning:~/.lightning/sparko-tls % openssl genrsa -out key.pem 2048
lightning@lightning:~/.lightning/sparko-tls % openssl req -new -x509 -sha256 -key key.pem -out cert.pem -days 3650
lightning@lightning:~/.lightning/plugins % chmod +x sparko_full_freebsd_amd64
lightning@lightning:~/.lightning/plugins % mv sparko_full_freebsd_amd64 sparko
lightning@lightning:~/.lightning/plugins % cd ~
you also need to create a configuration file for bitcoin-cli, a utility that communicates with bitcoind
lightning@lightning:~ % mkdir .bitcoin
lightning@lightning:~ % ee .bitcoin/bitcoin.conf
rpcconnect=192.168.0.1
rpcuser=test
rpcpassword=testcheck
lightning@lightning:~ % bitcoin-cli echo "test"
[
"test"
]launch lightningd
lightning@lightning:~ % lightningd --daemon
To say more about lightningd utility can be controlled lightning-cli, For example:
lightning-cli newaddr get address for new incoming payment
{
"address": "bc1q2n2ffq3lplhme8jufcxahfrnfhruwjgx3c78pv",
"bech32": "bc1q2n2ffq3lplhme8jufcxahfrnfhruwjgx3c78pv"
}lightning-cli withdraw bc1jufcxahfrnfhruwjgx3cq2n2ffq3lplhme878pv all send to the address all the money of the wallet (of all on-chain addresses)
Also commands for off-chain operations lightning-cli invoice, lightning-cli listinvoices, lightning-cli pay etc.
Well, for communication with the application, we have a REST Api
curl -k https://192.168.0.7:9737/rpc -d '{"method": "pay", "params": ["lnbc..."]}' -H 'X-Access masterkey'
To summarize
# jls
JID IP Address Hostname Path
1 192.168.0.1 bitcoind.space.com /zroot/jails/jails/bitcoind
2 192.168.0.2 tor.space.com /zroot/jails/jails/tor
3 192.168.0.3 nginx-rev.space.com /zroot/jails/jails/nginx-rev
4 192.168.0.4 paygw.space.com /zroot/jails/jails/paygw
5 192.168.0.5 webapp.my.domain /zroot/jails/jails/webapp
7 192.168.0.200 electrum.space.com /zroot/jails/jails/electrum
8 192.168.0.6 polipo.space.com /zroot/jails/jails/polipo
9 192.168.0.7 lightning.space.com /zroot/jails/jails/cln
We have a set of containers, each with its own level of access both from and to the local network.
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
zroot 279G 1.48T 88K /zroot
zroot/ROOT 1.89G 1.48T 88K none
zroot/ROOT/default 1.89G 17.6G 1.89G /
zroot/home 88K 1.48T 88K /home
zroot/jails 277G 1.48T 404M /zroot/jails
zroot/jails/bitcoind 190G 1.48T 190G /zroot/jails/jails-data/bitcoind-data
zroot/jails/cln 653M 1.48T 653M /zroot/jails/jails-data/cln-data
zroot/jails/electrum 703M 1.48T 703M /zroot/jails/jails-data/electrum-data
zroot/jails/nginx-rev 190M 1.48T 190M /zroot/jails/jails-data/nginx-rev-data
zroot/jails/paygw 82.4G 1.48T 82.4G /zroot/jails/jails-data/paygw-data
zroot/jails/polipo 57.6M 1.48T 57.6M /zroot/jails/jails-data/polipo-data
zroot/jails/tor 81.5M 1.48T 81.5M /zroot/jails/jails-data/tor-data
zroot/jails/webapp 360M 1.48T 360M /zroot/jails/jails-data/webapp-dataAs you can see, bitcoind takes up all 190 GB of space. But what if we need another node for tests? This is where ZFS comes in handy. With help cbsd jclone old=bitcoind new=bitcoind-clone host_hostname=clonedbtc.space.com you can create a snapshot and tie a new cell to this snapshot. The new cell will have completely its own space, but only the difference between the current state and the original will be taken into account in the FS (save at least 190 GB)
Each cell is its own separate ZFS dataset, and this is extremely convenient. do all sorts of other cool stuff like sending snapshots over SSH. We will not describe it, so there is already a lot.
It is also worth noting the need for remote monitoring of the host, we have for these purposes .
B - security
As for security, let's start from the key principles in the context of infrastructure:
Confidentiality - Standard tools of UNIX-like systems enforce this principle. We logically separate access to each logically separate element of the system - a cell. Access is provided through standard user authentication using the users' private keys. All communication between and to end cells is encrypted. Thanks to disk encryption, we can not worry about the safety of data during disk replacement or migration to another server. The only critical access is access to the host system, since such access generally provides access to data inside containers.
Integrity — The implementation of this principle occurs at several different levels. Firstly, it is important to note that in the case of server hardware, ECC memory, ZFS takes care of data integrity at the level of information bits out of the box. Instant snapshots allow you to backup at any time on the fly. Convenient cell export/import tools make cell replication easy.
Availability - It's optional. Depends on the degree of your fame and the fact that you have haters. In our example, we ensured that the wallet was accessible exclusively from the TOR network. If necessary, you can block everything on the firewall and allow access to the server exclusively through tunnels (TOR or VPN is another matter). Thus, the server will be cut off from the outside world as much as possible, and only we ourselves can influence its availability.
Impossibility of refusal - And it depends on further operation and compliance with the correct policies for user rights, access, etc. But with the right approach, all user actions are audited, and thanks to cryptographic solutions, it is possible to uniquely identify who and when performed certain actions.
Of course, the described configuration is not an absolute example of how it should always be, but rather one of the examples of how it can be, while retaining very flexible scaling and customization options.
But what about full virtualization?
About full virtualization using cbsd, you can . I'll just add that for work bhyve some kernel options need to be enabled.
# cat /etc/rc.conf
...
kld_list="vmm if_tap if_bridge nmdm"
...# cat /boot/loader.conf
...
vmm_load="YES"
...So if suddenly there is a need to start a docker, then we raise some debian and go ahead!
That's all
Perhaps this is all I wanted to share. If you liked the article, then you can throw bitcoins to me - . If you want to try the cells in action and have some bitcoins, then you can go to my .
Source: habr.com