The Intellectual Wilderness There is nothing more useless than doing efficiently that which should not be done at all.

2021.04.5 20:25

Erlang: Socket experiments preliminary to writing a web server from scratch

Filed under: Computing,Science & Tech — Tags: , , , , , , , — zxq9 @ 20:25

A relative newcomer to networking in Erlang, Dr. Ajay Kumar, has started a self-educational project to create a web server from scratch in Erlang to give himself some first-hand insight into how TCP sockets work in Erlang and how web servers work in general. Web servers are a thing almost everyone has written against or for, but few have tried to implement on their own because socket programming sounds scary and time consuming to learn.

This video is quite short but incidentally demonstrates how not scary socket programming is and how easy it is to experiment with networking concepts on your own. Go experiment! Write networky things! It’s fun!

As an aside to this… I have a two-part explanation video that explains everything that is going on inside the service at he is basing his web server on. In the first part I explain what is going on within the default chat server that ZX templates as a network service project (kind of slow, covers basics for Erlang and ZX newcomers), and in the second part I explain how I used that as a basis for creating a telnet chat service that implements global shouts, channels, permissions, and other basic features needed for a real chat service (a bit better paced if you already know your way around Erlang and discusses some higher-level concepts such as service structure and the “service -> worker pattern“).

2020.11.22 19:20

Erlang: Building a Telnet Chat Server from Scratch Using ZX

Filed under: Computing — Tags: , , , , , , , — zxq9 @ 19:20

A few weeks ago I made a two-part video discussion about building a telnet chat server from scratch using ZX and forgot to post any prose reference to it. (Most people are following my blog RSS, not my (extremely tiny) video channels.)

The resulting project is called “Trash Talk” and it has a repo on GitLab here. The license is MIT and is quite simple to hack around on, so have fun.

Part 1

The first video is a bit slower paced than the second. I cover:

  • What ZX templates when you do zx create project and select the “Traditional Erlang Application” project option
  • How everything fits together and works
  • Discussion about why things are structured the way they are in OTP applications
  • Demonstrate a little bit of command implementation in the telnet server (mostly to show where in the code you would do that).

This video is a lot less exciting than the second one because there aren’t a lot of jazzy features demonstrated, but it is probably the one you want to watch once now if you’re new to Erlang and a second time a few weeks later once you’ve written a few projects and stubbed your toes a few times (a second viewing brings different insights, not because the video changes but because you change through your experiences).

Part 2

The second video is the more fun one because the initial explanation that covers the critical question of “Why does anything do anything?” has already been covered in the first one, and while you might find the first video interesting, it isn’t as exciting as this second one where you get to see features that do stuff that might be more relevant to problems you have in your real-world projects get implemented. In this one I cover:

  • Checking the state of the running system using observer
  • The difference between zx and zxh when running your code in development
  • The “Service ⇒ Worker Pattern” (aka “SWP”) and how ZX can help you drop one in via zx template swp
  • One way to implement text commands in a structured way, with categories of commands indicated by prefixes (this seems elementary and very IRC-ish at first, but in command protocols a direct parallel of this often happens at the byte level — so think on that)
  • How to separate the “clients” concept as a service from the “channels” concept as a service, and discuss how that idea extends to much more complex systems
  • A bit more useful version of a “service manager” process in the form of the chan_man
  • And much much more! (advertising people from the 1980’s tell me I should always say that instead of “etc.”)

These aren’t super jazzy, but they get the job done. Hopefully they give some new Erlanger out there a leg-up on the problem of going from a “Hello, World!” escript to a proper OTP application.

2011.05.25 21:40

tl;dr Instructions for Drupal7 on rpm-based Systems

Filed under: Computing — Tags: , , , , , , — zxq9 @ 21:40

I threw a few sites up for folks in need the other day, one of them being myself (lots of folks’ businesses were wrecked during the tsunami two months ago). From doing that I realized the Drupal 7 documentation on rpm-based systems is a bit lacking (especially in Japanese). There are a few reasons for this. For one thing, Drupal 6 is still what’s in the Fedora repositories (at least as of F15 — and its orphaned?!?). So the Fedora wiki has basic install instructions for 6, but not 7. The Fedora-ized version also places things in a symlinked area under /usr, which may be more secure, but it confuses new Drupal users about “how to get rid of the” problem under Apache. And… lastly, SELinux requires a few adjustments to let httpd and drupal to work well together. Most users’ reaction to the first sign of SELinux issues is to hurriedly turn it off… emphasis on the hurr in “hurriedly” there, because that is stupid.

Even a lot of professional web developers do this, which should drive home the point I’ve made elsewhere of web people != systems people (though systems people might be capable web developers). As of this writing Ubuntu doesn’t even field SELinux by default and their millions of users don’t get chewed up as easily as Windows folks do, but the majority use case for Ubuntu is not (and should not, imo) be server deployment. But why risk it when you have such a powerful security tool right in front of you? For the enterprise I just don’t find it prudent to abandon such a great and easy tool. That’s like choosing to not learn iptables — which some folks have also opted out of as well.

So, without further ado, here is the quick and dirty to get Drupal 7 working on a Fedora/CentOS/RHEL type OS with SELinux intact:

[Please note these instructions assume three things: 1) a completely fresh minimal install, 2) you have control of the server, and 3) you are able to execute commands as root or through sudo. Also note that I have removed foreign language setup from this, as I doubt anyone who reads my blog really needs Japanese but me.]

[root@localhost]# yum install postgresql postgresql-server php-pgsql php-xml \
    php-pear php-devel gcc zlib-devel libcurl-devel make wget httpd php-mbstring
[root@localhost ~]# cd /var/www/html
[root@localhost html]# wget
[root@localhost html]# tar -zxf ./drupal-7.0.tar.gz
[root@localhost html]# mv drupal-7.0 drupal7
[root@localhost html]# cd drupal7
[root@localhost drupal7]# pecl install pecl_http
[root@localhost drupal7]# pecl install uploadprogress
[root@localhost drupal7]# echo > /etc/php.d/php_http.ini
[root@localhost drupal7]# echo  >> /etc/php.d/php_http.ini
[root@localhost drupal7]# service postgresql initdb
[root@localhost drupal7]# service postgresql start
[root@localhost drupal7]# chkconfig postgresql on
[root@localhost drupal7]# chkconfig httpd on
[root@localhost drupal7]# setsebool -P httpd_can_network_connect_db=1
[root@localhost drupal7]# setsebool -P httpd_can_sendmail=1
[root@localhost drupal7]# setsebool -P httpd_unified=1
[root@localhost drupal7]# cp sites/default/default.settings.php sites/default/settings.php
[root@localhost drupal7]# chmod 666 sites/default/settings.php
[root@localhost drupal7]# su postgres
bash-4.1$ createuser --pwprompt --encrypted --no-adduser --no-createdb drupal
Enter password for new role:
Enter it again:
Shall the new role be allowed to create more new roles? (y/n) n
bash-4.1$ createdb --encoding=UNICODE --owner=drupal drupaldb
bash-4.1$ exit
[root@localhost drupal7]# cp /var/lib/pgsql/data/pg_hba.conf /var/lib/pgsql/data/pg_hba.conf.original
[root@localhost drupal7]# vi /var/lib/pgsql/data/pg_hba.conf

Add the following line at about line 71 or so, just after the local all all ident line (check first, don’t blindly dump this in with sed because this could all be wrong if you’re running a different version of Postgres or reading this far in the future):

host    drupaldb    drupal    md5

Remove the Apache and OS identification tags on server-generated error messages (such as the default Apache ### error messages):

[root@localhost drupal7]# vi /etc/httpd/conf/httpd.conf

Replace “ServerTokens OS” with “ServerTokens Prod
Replace “ServerSignature On” with “ServerSignature Off

And since I’m paranoid and use my servers only as servers (and prefer to send logs to a separate logging server), I also change “LogLevel Warn” to “LogLevel Info” and let my parsing scripts do the work of finding the important stuff. That makes forensics a lot easier later on down the road (though more compute intensive).

Add the following lines to your httpd.conf file if you are running multiple websites on a single server (on a single IP address that is hosting multiple domain names):

NameVirtualHost *:80

<VirtualHost *:80>
    DocumentRoot /var/www/html/drupal7
    ErrorLog logs/

<VirtualHost *:80>
    DocumentRoot /var/www/html/

If you are not running virtual servers, or if you are running them in /home/user/public_html or whatever, adjust the way the file is written.

Now input iptable rules necessary to open port 80 for web traffic from outside, and allow httpd to access Postgres:

[root@localhost drupal7]# iptables -vI INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
[root@localhost drupal7]# iptables -vI INPUT -m state --state NEW -m udp -p udp --dport 5353 -j ACCEPT

Now point a browser at the server and do your Drupal installation setup. Once you’re done there reset the permissions on sites/default/settings.php and remove the last iptables rule:

[root@localhost drupal7]# chmod 644 sites/default/settings.php
[root@localhost drupal7]# iptables -vD INPUT -m state --state NEW -m udp -p udp --dport 5353 -j ACCEPT

If everything went correctly you should be able to use Drupal 7 with SELinux in Enforcing mode, with your iptables intact aside from however you remote login for administration (SSH, if you use it, if you don’t, then close off port 22 and do chkconfig sshd off as well).

The next time you reboot you will notice you can still log in to your shell, but you can’t access the website with a browser. That is because the iptables rule fell off (they don’t persist unless you tell them do). Once everything works the way it should, commit the iptable rule that is letting port 80 stay open:

iptables-save > /etc/sysconfig/iptables

And while we’re messing with security… let’s go ahead and turn off a php feature that the cracker wannabes have recently learned about: allow_url_fopen.

[root@localhost drupal7]# vi /etc/php.ini

Replace “allow_url_fopen On” with “allow_url_fopen Off

It is very unlikely that you will need all_url_fopen to be active, as very few modules use it (there is a Drupal-sepecific alternative to this, so it seems). Of course, if your site breaks it would be good to check if you actually did need this, but otherwise I’d leave it turned off until things go wrong.

Now we need to restart Apache:

[root@localhost drupal7]# apachectl restart

Given that the biggest fans of tutorials such as these are the sort of folks who would never spend the time to research the meaning of all this themselves and given that you wouldn’t be reading this if you already knew how to do the above in the first place, I must remind you to head to your favorite search engine and do searches for things like “hardening sshd”, “hardening httpd”, “turning off root login”, “using public key encryption with sshd” and anything else that might strike your fancy (protip: read as much as you can about SELinux and iptables).

Powered by WordPress