AWS Elasticbeanstalk makes it very simple to create an environment with load balanced instances running a web server proxy in front of your custom app. This is as easy as a couple of clicks from the web interface, or just
eb create &
eb deploy from the Elastic Beanstalk Command Line Interface (EB CLI).
As long as you aren’t serving WebSocket, this all just works right out of the box without any custom configuration. That is a pretty powerful ops setup for a couple of commands! However, if you want to serve WebSocket traffic, you’ll need to edit the configuration of your proxy server (Nginx in my case) to allow the http upgrade.
Previously, you also had to configure you Elastic Load Balancer to get around the fact that it didn’t really support WebSockets. But, the new Application Load Balancers (now available on Elasticbeanstalk environments) have native support for WebSockets.
But, to support WebSockets between your Apache/Nginx Web server proxy and your app, you might need to edit your Web server’s configuration file based upon what Elasticbeanstalk provides as a default template. At least this was the case for me using the provided default Nginx configuration.
The popular posts I found floating around on how to modify your Elasticbeanstalk Nginx configuration suggest using the .ebextensions (as described in my last post) with a
files section to create a custom Nginx.conf file that ends up completely replacing the basic Nginx configuration that Elasticbeanstic provides in
This approach amounts to creating a config file in
.ebextensions with content like:
..and then a
commands section to either remove or rename the auto generated
00_elastic_beanstalk_proxy.conf file so that the new file is used instead.
While this approach works fine, the problem I have is that it amounts to including a whole new copy of an Nginx configuration, hardcoded into a separate custom configuration section and ignoring what AWS Elasticbeanstalk provides as a template. Essentially, the only thing we really need to replace is
proxy_set_header Connection ""; in the default
lines #26-27 from above.
Whenever I make configuration changes to default provided files, like the Elasticbeanstalk provided
00_elastic_beanstalk_proxy.conf, I prefer an approach that respects future changes that Elasticbeanstalk might make to its default configuration decisions. In other words, if we are replacing the entire file with our very slightly modified version, we prevent our instances from ever getting changes that AWS might decide are important for its Elasticbeanstalk setups. This might be things related to default interaction between its load balancers and web servers, time outs, health checks, etc.
If possible, I prefer an approach that respects future changes automatically instead having to remember a year or two from now that “oh yeah, I’m replacing that file on every deployment and that’s why my server isn’t automatically working with those new load balancers.”
The approach I decided to take was figuring out how to use the default
00_elastic_beanstalk_proxy.conf file as is, but just replace the:
proxy_set_header Connection "";
with what I needed to allow WebSockets (i.e. HTTP upgrades):
proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade";
First, just to make sure I was on the right path, I ssh’d into my instance and manually edited the
00_elastic_beanstalk_proxy.conf file, restated Nginx and sure enough WebSockets connections were flowing. But, I discovered that upon next deployment my changes to
00_elastic_beanstalk_proxy.conf were overwritten.
After a little hunting around, I discovered that Elasticbeanstalk uses templates located in:
to build their default configuration files. In this case, the template I was looking at was:
Sure enough, right in the header of that file, it gives me a tip on how to proceed:
1 2 3 4 5 6 7 # Elastic Beanstalk managed configuration file # Some configuration of nginx can be by placing files in /etc/nginx/conf.d # using Configuration Files. # http://docs.amazonwebservices.com/elasticbeanstalk/latest/dg/customize-containers.html # # Modifications of nginx.conf can be performed using container_commands to modify the staged version # located in /tmp/deployment/config/etc#nginx#nginx.conf
According to the comment, it seems I should use
container_commands in order to replace the standard
proxy_set_header Connection ""; in the
/tmp/deployment/config/#etc#nginx#conf.d#00_elastic_beanstalk_proxy.conf with the couple of lines I need for for WebSockets.
To do that, I created a file called
nginx_websocket_upgrade.conf (name was arbitrary) in
.ebextensions with the following content:
1 2 3 4 5 6 7 container_commands: 01_nginx_websocket_support: command: | sed -i '/\s*proxy_set_header\s*Connection/c \ proxy_set_header Upgrade $http_upgrade;\ proxy_set_header Connection "upgrade";\ ' /tmp/deployment/config/#etc#nginx#conf.d#00_elastic_beanstalk_proxy.conf
And on the next
eb deploy WebSocket connections were flowing!
To verify that it was actually the result of this change that allowed WebSockets to start flowing, I did a quick
eb ssh and took a look at
/etc/nginx/conf.d/00_elastic_beanstalk_proxy.conf, and verified that it contained the changes that the new
container_command made to the template file.
Hat tip on getting the sed command right by finding this gist here: https://gist.github.com/adamgins/f99635447a1239289460 that included the syntax to replace the respective proxy_set_header.