🤔 Migrating a Dockerised AngularJS app with Haskell API on AWS ECS... to Elm with Yarn and Webpack plus Serverless Haskell on AWS Lambda, API Gateway and CloudFront SSL
Having now won “least readable title ever”, let’s, erm, begin…
Previously on Sloganator…
Recently I released v2.0 of long-standing pet irony web-app, The Sloganator. You know, this one:
Since its birth in 2015, I’d had various suggestions for tweaks and improvements mainly around content. A separate thread (one thread of irony is insufficient) was its ongoing use of Shiny New Things, which involved me deliberately trying (too?) many new technologies at once.
To someone who at the time knew very little Haskell, this was probably the biggest ask, and navigating Snap Framework wasn’t easy (it seems good if slightly unmaintained). The infrastructure / Dockerising side of it was more familiar, but itself presented some good challenges in the Haskell world.
So several years later, emboldened by the “success” of this story, and definitely a bit poorer after keeping EC2 instances alive for the ECS, I realised that AngularJS was so 2014, and Docker was everywhere anyway now, and Snap hadn’t taken off. Hmm.
Meanwhile, Elm as a front-end language and architecture was looking very interesting (EDIT: this was a blissfully before the 0.19 controversy if you care about stuff like that).
The potential to move to a more trendy architecture (Serverless / Functions As A Service etc) was noted too.
Front end & API (same container)
- FullPage.JS & Materalize CSS
- AngularJS & JQuery
- HTML 5 / CSS 3
- Snap Framework
- Haskell / GHC 7
- Debian Linux
- ECS (managed via Elastic Beanstalk)
- AWS EC2
- FullPage.JS / Materalize CSS
- HTML 5 / CSS 3
- Cloudfront (CDN)
- Haskell function(s) (GHC 8.02)
- Serverless & Serverless Haskell shim
- AWS Lambda (Node 6.x)
- AWS API Gateway
Getting from A to B
This took a long time, and maybe wasn’t the optimal order. In fact it was long enough ago now that I’m not sure it was the order, but it seems like a good order:
- The front-end: add a front end (
/web, I called it) for Elm (0.18).
- …for which we’ll need a package manager…
- Remove AngularJS code (there wasn’t much) from the JS layer.
- Remove Snap (high-performance web framework for Haskell)
- So move hosting static front-end (Elm SPA + assets) onto S3 (we no longer have a webserver…)
- Add Serverless framework (for which we’ll need npm on the root project, and
- Write Serverless handlers, and integrate with AWS API Gateway.
- …upgrading Stack build along the way to get newer dependencies
- Configure SSL support with CloudFront and Amazon Certificate Manager
- Rethink DNS structure to allow API to be hosted on SSL too.
- Switch DNS over
I wanted to keep the one repo if possible, though there’s logic that would argue not to do this. In the end I settled for
web/ being a self-contained Elm app under the main project.
This remains confusing as Serverless is a Node / npm project, meaning there’s a
package.json at the root level, as well as one in the
web/ folder for the Yarn-ified Elm build. I could probably unify these one day.
I think of it as a “cut down, friendlier Haskell for the web”. This took longer to get the hang of than I thought, and even then I haven’t really got the hang of it. Enough to get it working, and I’m definitely a fan so want to carve out some time to explore larger scale apps. (Update: I’ve built another PoC. Still confusing, still love it).
Building that Elm
In keeping with my original Sloganator philosophy (try all the new things at once, and see what happens), I went with
better more battle-hardened build and test tooling from the outset. That means no boilerplate generators and ditching Elm’s own preview servers.
I’m a fan of Yarn. Not everyone is, but I like it – or at least dislike npm, but again the point is it’s newer and shinier and has a beautiful colourful CLI… so I’m sold 😆
Webpack, meanwhile, is one of those everyone’s-using-it JS things which still feels a bit magical to me. Luckily there’s an official-ish section on Webpack and Elm.
I ended up with plugins (other than the normal loaders include the magical
clean-webpack-plugin– why this isn’t default I don’t know.
html-webpack-plugin– this way allowed templating the SPA entry point page (with EJS), controlled by Webpack and thus the build process.
mini-css-extract-plugin– seemed quite useful for CSS minification and separation to a file for Production (and… sanity).
Getting the interop right involved getting rid of as much jQuery as possible in the end, and just using Elm’s messages. Whilst this feels right (especially now Github did it!) Sometimes this just isn’t an option though – e.g. when plugins are in heavy use.
The site uses FullPage.js for the, err, full page interaction, and this was built as a JQuery plugin.
🆕 jQuery – gone 😄. It wasn’t too hard, latest FullPage (paid version now 😧) and MaterializeCSS now both support JQuery-free life.
A few things had to use
element.addEventListener but it’s really not so bad.
Ports are all part of the magic of Elm. One port was only needed here (there’s no JS -> Elm at all either) – the TTS, which uses the little-explored Web Audio API that modern browsers support to varying degrees.
The trigger to the whole migration was the combination of using Serverless more at work, and finding the serverless-haskell project on Github.
Other than the Haskell plugin, there were a couple of others useful for the SSL and front-end parts.
serverless-domain-manager came in handy to help configure CloudFront and AWS Certificate Manager a bit, but in reality there was a fair amount of hand-holding.
serverless-finch proved perfect for the deployment of static assets to S3 as part of the – just needed directing to the
web/dist/ directory (and a bucket).
Realising this could and does work well enough (it uses a Node base, wraps Stack at build time, and calls the compiled binary process at runtime, with
stdout as the interface – nice and UNIXey I guess). Under the hood, the build is done with Docker (fpco/stack-build in fact. Warning: it’s ~3GB of image). I guess this keeps things sane given how hard GHC builds can be cross-platform.
Configuring the function
Using AWS API Gateway was sensible, and
lambda-proxy was the Serverless “integration” method I went for in the end, after reading (then ignoring) the confusing API GW Serverless documentation.
serverless.yml config in the end, though getting there never is:
To help serve the API Gateway endpoint, it seemed a good time to use ADTs to define the possible variants of the API.
import Sloganator.GymSlogans import Sloganator.HipsterSlogans import Text.Read (readMaybe) import Data.Char (toLower) data SloganType = Gym | Hipster deriving (Show, Eq) -- TODO: work out a less clunky way of doing this instance Read SloganType where readsPrec _ s = case toLower <$> s of "gym" -> [(Gym, "")] "hipster" -> [(Hipster, "")] _ ->  sloganHandler :: Maybe Text -> IO (Maybe String) sloganHandler maybeVariant = -- We can fail the read (Nothing) or have been _supplied_ a Nothing -- ...so bind that Maybe, baby. let variant = maybeVariant >>= (readMaybe . T.unpack) in case variant of Just Gym -> Just <$> gymSloganator Just Hipster -> Just <$> hipsterSloganator _ -> return Nothing
Dealing with content type negotiation became a bit of a pain. The original contract was simple enough: a plain text request should elicit a plain-text API response, but with
Accept: application/json, we want a minimally JSON-wrapped response, with an appropriate JSON
import AWSLambda.Events.APIGateway import Control.Lens import qualified Data.HashMap.Strict as HashMap import Data.Text (Text) import qualified Data.Text as T import Data.Semigroup ((<>)) import Data.Aeson (encode) import Data.String.Conversions (cs) -- | Convenience newtype ContentType = ContentType Text jsonType = ContentType "application/json" plainText = ContentType "text/plain" -- | Form an HTTP response appropriate for an optional content-type and slogan responseForSlogan :: Maybe ContentType -> String -> APIGatewayProxyResponse Text responseForSlogan maybeContentType slogan = case maybeContentType of (Just (ContentType "application/json")) -> okWithContentType jsonType json where json = cs $ encode [slogan] _ -> okWithContentType plainText (T.pack slogan) -- | A 200 OK reply for the Gateway, with content type and body as passed in okWithContentType :: ContentType -> Text -> APIGatewayProxyResponse Text okWithContentType (ContentType contentType) body = responseOK & responseBody ?~ body <> "\n" & agprsHeaders .~ HashMap.fromList [ ("Access-Control-Allow-Origin", "*") , ("Content-Type", contentType) ]
The API Gateway Handler
If you go down the API Gateway route (I soon did), the docs give some hints / examples.
Here’s what I ended up with (recently refactored for neatness):
sloganGatewayHandler :: APIGatewayProxyRequest Text -> IO (APIGatewayProxyResponse Text) sloganGatewayHandler request = do putStrLn $ "Got request: " <> show request let maybeContentType = ContentType <$> HashMap.lookup "Accept" (request ^. agprqHeaders) variant = HashMap.lookup "variant" (request ^. agprqPathParameters) slogan <- sloganHandler variant return $ maybe responseNotFound (responseForSlogan maybeContentType) slogan
This is now a dream 😁
It takes seconds (previously the Dockerised solution(s) weren’t fast), and works every time – Serverless (and the Haskell plugin) is more reliable than I was expecting.
Some lessons learned, or at least pain points (that I remember – it was a while ago).
As part of the regular releases in the last few years I’d tried to keep up with Stackage LTS releases on update, and GHC itself from 7.8 up to 8.2.2 now. With this of course the controlled explosions of major version bumps, but I managed to sort most of these out easily even given limited experience doing so before.
Binaries are large(ish)
- I found ~30MB for a simple(ish) Haskell function. Understandable, considering the amount of libraries GHC is compiling, but not for you if you’re on a minimalist tip.
SSL with DNS
- To get DNS to play ball at all, I had to do some breaking changes at the URL level
https://api.sloganator.iois now the root for all API requests
https://sloganator.ionow serves content over SSL
- A fair bit of this isn’t automated – the SSL certificate was fairly manually created, for example, but the second time around I’d hazard more could be automated from the outset.
The big payback
The great news is API Gateway, Lambda, S3, and CloudFront all take a lot of load before costing much and there’s not much more to it at runtime.
Here are the costs on that AWS account (the “other” EC2 cost there is one instance for tfl.declension.net – which needs rewriting too…) :
So quite apart from the learning aspect, the cost for The Sloganator has gone from around $12 / month (EC2 + EB / ECS) to a under a dollar per month for the Serverless version. 💸