Run Tensorflow scripts from Node.js server deployed on AWS as Docker container

Machine and deep learning tooling is excitingly accessible and fun for a developer to work with. There are couple of ways to develop/play with machine learning code:

There might be more options I am unaware of or more likely I just cannot find them using simple search engine queries.

I tried couple of those above - Jupyter notebook, scripting locally and Kaggle. The approach you take usually depends on what you want to achieve. If you want to learn then starting with the cloud solution is the shortest way which gives the most pleasure, picking Kaggle could probably be one of the best choices, you can see what others do and can fork their kernels and learn by example.

Whilst playing/learning with machine learning is undeniably a great way to spend your time, it is a bit more exciting to use the skills and build a production ready model for others to consume. For this choice now I pick scripting - writing Python scripts which are later deployed to production and exposed via browser interface.

Note: if you prefer looking into code then check out an implementation supporting this article in Github repo ivarprudnikov/char-rnn-tensorflow


  • Prepare Python scripts for training & generating
  • Store training data & training parameters
  • Train models in browser interface
  • Generate text based on previously trained model
  • Wrap implementation in Docker
  • Use CI service to build Docker image for the registry
  • Run Docker image on AWS and expose it to the WWW

Text generator

As it happens there are examples of simple text generators in the web. One of them leads me to a character generator detailed in Andrei Karpathy’s blog post “The Unreasonable Effectiveness of Recurrent Neural Networks (RNN)” published in mid 2015. This RNN example relieves me from any explanations how/why it works and is generally concise in its implementation.

Part of the problem is solved but current article mentions Tensorflow and Karpathy’s implementation is written in Torch. For this reason there was a reimplementation of char-rnn I found in another Github repository hzy46/Char-RNN-TensorFlow which I forked and tweaked a bit.


Happily char-rnn-tensorflow is capable of training the model by passing it a text file as an argument amongst other possible options.

python \
  --input_file data/shakespeare.txt  \
  --name shakespeare \
  --num_steps 50 \
  --num_seqs 32 \
  --learning_rate 0.01 \
  --max_steps 20000

Above will produce and save couple of Tensoflow checkpoints but will take some time. Training time varies depending on hardware you have at disposal. Lengthy training process forces us to use asynchronous API when exposing this feature via HTTP so that user gets notified after s/he submits request to train on given data and process finishes in the background.

Generating text

After training completes and checkpoints were saved we can call another script to generate text for us:

python \
  --converter_path model/shakespeare/converter.pkl \
  --checkpoint_path model/shakespeare/ \
  --max_length 1000

Above script needs a path to checkpoint data to pull variable values from it and use in the rebuilt model, it also uses the vocabulary file which was dumped in the process of training, that includes characters used for text generation.

Web application

Given we have python scripts for training and character generation it is possible to wrap those in a simple app exposed to the public internet. User will be able to upload and train her own sample then generate some text after training finishes.

To build an app I’ll use Node.js with Express framework. There will be couple of publicly accessible paths to deal with uploading of training data, training of that data and generating text out of it. User will also be able to see other submissions as we do not care about security and accounts to make this exercise simpler.

Uploading data

Simplest approach to solving the upload problem is a basic html form. User should have UTF-8 encoded plain text file available on her system. And app should be able to render the form and process form POST with multipart/form-data. Ideally uploaded plain text files could be stored somewhere on AWS S3 or similar services to leverage almost infinite scalability but for this exercise I’ll keep those on the same server running application and executing Python scripts.

Render upload form

Basic knowledge of Express framework here is assumed. Below example expects view engine along with default views directory set to be able to render html. If you follow code in git repository it will be a bit different and will have more features used.

router.get('/upload', (req, res) => {
  // render upload.html/upload.ejs file
<!-- excerpt from upload html/ejs file -->
<form action="/upload" method="post" enctype="multipart/form-data">
    <legend>Training data</legend>
      <label for="customFile">Choose training data file</label>
      <input name="file" type="file" id="customFile" required>
      <small class="form-text text-muted">
        File should be UTF-8 plain/text containing data you want to use for training.
  <button type="submit">Upload</button>

Process form POST

In order to process multipart form request I chose to use Busboy dependency.

$ npm i -S busboy

Original filename is not being used and instead is replaced with train.txt which is going to be the same for every upload. In order to distinguish those files they will be living in separate directories named after generated id which is a timestamp here in the example. After successful upload user will be redirected.'/upload', (req, res) => {

  // generate id
  const id =
  // prepare dependency used to process request
  const busboy = new Busboy({
    headers: req.headers,
    limits: {
      fileSize: 1024 * 50, // bytes
      files: 1 // only one file per request
  let fileStream = null
  let filePath = null
  let folderPath = path.join('uploads', id)

  // handle multipart file
  busboy.on('file', (fieldName, file, fileName) => {
    if (fileName) {
      filePath = path.join(folderPath, 'train.txt')
      fileStream = file.pipe(fs.createWriteStream(filePath))
  // redirect on success, otherwise render same page with error message
  busboy.on('finish', () => {
    res.set({Connection: 'close'});

    if (!fileStream) {
      res.render('upload', Object.assign(res.locals, {
        error: "Cannot save given training data"
    } else {
      fileStream.on('finish', async () => {
      fileStream.on('error', () => {
        res.render('upload', Object.assign(res.locals, {
          error: "Error occurred while saving file"
  // pipe request stream to our dependency

One problem is almost solved, training file is ready to be used.


We could store everything in the filesystem but eventually it gets quite complicated. Initially I thought example without database will be a bit more readable but the effect was opposite as soon as I wanted to render more details in html. It is useful to track when user uploads data, when training starts and ends, even giving names to those training jobs is useful, but storing them in filesystem seemed a bit dull and verbose to implement.

Apart from those mentioned useful parts it will be necessary to store logs as well. Logs are going to be produced when training on uploaded data. I chose to store those in database after reading quite old Travis blog post Solving the Puzzle of Scalable Log Processing.


There are 2 things I want to store:

  • model represents training job and contains details such as id, time it was created, training parameters, if user uploaded text data, is training complete, etc.
  • log is part of model but instead of having one big field containing all the text it will be split into lines which will make it easier to insert new data.


Now that there is a relationship between models and structure is known in advance - will not likely to change, it is sensible to choose relational database. I believe that lowest common denominator will be MySQL. Document store such as Mongo does not really make much sense here and not only because of relationship but more due to the nature of log data which will drip line by line.

create table model (
  id             varchar(255) not null,
  created_at     timestamp             DEFAULT CURRENT_TIMESTAMP,
  updated_at     timestamp             DEFAULT CURRENT_TIMESTAMP
  name           varchar(255) not null,
  train_params   json         not null,
  has_data       tinyint      not null default false,
  is_in_progress tinyint      not null default false,
  is_complete    tinyint      not null default false,
  training_pid   varchar(255),
  primary key (id)
  COLLATE utf8_general_ci;

alter table model
  add constraint unique_id unique (id);
alter table model
  add constraint unique_pid unique (training_pid);

create table model_log (
  model_id varchar(255) not null,
  position int          not null,
  chunk    text         not null
  COLLATE utf8_general_ci;
alter table model_log
  add index FK_MODEL_LOG_MODEL (model_id),
  add constraint FK_MODEL_LOG_MODEL foreign key (model_id) references model (id);

A quite simple schema encapsulates what I mentioned before, table model will contain metadata about training and model_log will contain chunks of log text. Position in model_log ins necessary in theory to guarantee order of log entries.

In order for Node.js to connect to database we’ll need to get relevant dependency:

$ npm i -S mysql

To verify that connection works we could try listing models:

const mysql = require('mysql')
const pool = mysql.createPool({
  connectionLimit: 10,
  connectTimeout: 20 * 1000,
  acquireTimeout: 20 * 1000,
  timeout: 10 * 1000,
  host: process.env.MYSQL_HOST || 'localhost',
  user: process.env.MYSQL_USER || 'root',
  password: process.env.MYSQL_PASSWORD || '',
  database: process.env.MYSQL_DATABASE || 'rnn_generator',
  port: process.env.MYSQL_PORT || 3306,
  // ssl: "Amazon RDS", will be necessary later

pool.query("select * from model order by updated_at desc limit ? offset ?", [10, 0], (error, rows) => {

Training parameters

Just before being able to run those scripts we need to pass some arguments to them as well. Before that user ought to be able to tweak them in UI. Arguments are going to be exposed as a web form to the user and upon POST those values will be stored in the database. You can see all of training options used in a web form in the git repository. As an example a form looks similar to the following:

<form action="/<%= %>/options" method="post" enctype="multipart/form-data">

  <h1 class="h3 text-center mb-3">Update training options</h1>

  <% if(locals.errors){ %>
    <p class="alert alert-warning">
      Form contains errors
  <% } %>

    <legend>Training options</legend>

    <div class="form-group row">
      <label for="num_seqs" class="col-sm-6 col-form-label">Number of seqs in one batch</label>
      <div class="col-sm-6">
        <input name="num_seqs" placeholder="Default: 32" type="number" id="num_seqs"
               class="form-control <% if(fieldErr("num_seqs")){ %>is-invalid<% } %>"
        <% if(fieldErr("num_seqs")){ %>
        <div class="invalid-feedback"><%= fieldErr("num_seqs") %></div>
        <% } %>
    <!-- More fields in Github repository -->
    <hr class="mt-5">
    <div class="text-right">
      <a href="/model/<%= %>" class="btn btn-outline-secondary">Cancel</a>
      <button type="submit" class="btn btn-primary">Update</button>

Above you can see I am using variable interpolation <%= variable %> which is part of ejs templating engine I have configured Express app with. This form expects model to be passed to renderer, the one returned from MySQL and data which holds form field values, it also expects helper functions fieldData() which is a shorcut to access values in data object and fieldErr() which checks if there is an error for a given form field. Rendering of the above looks like:

router.get('/:id/options', checkPathParamSet("id"), loadInstanceById(), (req, res) => {
  res.render('training_options', Object.assign(res.locals, {
    data: JSON.parse(req.instance.train_params),
    model: req.instance

Here checkPathParamSet and loadInstanceById are helper middleware functions that are reused amongst other methods as well:

function checkPathParamSet(paramName) {
  return (req, res, next) => {
    if (!req.params[paramName]) {

function loadInstanceById() {
  return async (req, res, next) => {
    let instance = await db.findModel(
    if (!instance) {
    } else {
      req.instance = instance;

Schema validation

Storage of training parameters is pretty much straightforward except their validation. To make sure we comply with Python script api those requirements need to be formalized somewhere else but script itself and then validated against before storage. This could also be an ad hoc implementation but validation ought to be reused before calling script as well, also reading code is harder that looking into schema. I am using JSON schema for parameter definition and validation, below example shows only couple of fields:

  "$id": "generator/schema/training/options.json",
  "$schema": "",
  "type": "object",
  "properties": {
    "num_seqs": {
      "type": "integer",
      "minimum": 1,
      "description": "number of seqs in one batch"
    "num_steps": {
      "type": "integer",
      "minimum": 1,
      "description": "length of one seq"
  "additionalProperties": false

There are couple of JSON schema validators in the wild but I chose ajv as it supports latest schema drafts. Lets see how validation logic is implemented:

const Ajv = require('ajv')
// above schema sample
const trainOptionsSchema = require("train_arguments_schema.json") 
const ajv = new Ajv({allErrors: true, coerceTypes: true, removeAdditional: true})
const validator = ajv.compile(trainOptionsSchema)

function chackTrainParams(params) {
  if (validator(params)) {
    return null
  let errors = {}
  // validator shows path of failing leaf starting with a dot
  // changing it to simplify rendering of error messages
  validator.errors.forEach((err) => {
    let keyWithoutTrailingDot = err.dataPath.replace(/^\./, "");
    errors[keyWithoutTrailingDot] = err.message
  return errors

With above implementation it is easy enough to check if there are any errors in current object:

chackTrainParams({num_seqs: 0})

// returns
// { num_seqs: 'should be >= 1' }


Previously, for document upload, I used Busboy but when using it with form fields it is a bit verbose, gladly there is a wrapper around that dependency called multer which puts form fields into req.body. For just the form field submissions use multerUpload.none() middleware.'/:id/options', checkPathParamSet("id"), loadInstanceById(), multerUpload.none(), asyncErrHandler.bind(null, async (req, res) => {

  let model = req.instance

  // filter out empty values
  let params = Object.keys(req.body).reduce((memo, val) => {
    if (req.body[val] != null && req.body[val] !== "") {
      memo[val] = req.body[val]
    return memo
  }, {})

  let errors = chackTrainParams(params)
  if (errors) {
    return res.render('training_options', Object.assign(res.locals, {
      error: "Found " + Object.keys(errors).length + " errors",
      errors: errors,
      data: params,
      model: req.instance

  await db.updateModel(, {
    train_params: JSON.stringify(params)


Another thing above is async error handler which ought to be explicitly written as until now Express does not allow handling them:

const asyncErrHandler = (asyncFn, req, res) => asyncFn(req, res)
  .catch(err => {
    console.log((new Date()).toISOString(), "[ERROR]", util.inspect(err))

Running python scripts

All ingredients are ready to be used, training data will be uploaded to a location on disk, training parameters will be stored in a database. It is now necessary to use those details and run Python scripts which were forked already. I’ve written about how to run Python scripts from within Node.js application already so will not delve into much details, make sure to skim it through though “Using Python scripts in Node.js server”.


First I need action buttons in UI which will start/stop training process, upon click those should hit appropriate path handler in application which will do the rest. Those buttons will be exposed only in certain conditions - start when training data is uploaded and stop only when training is in progress:

<% if(model.is_in_progress){ %>
  <form class="d-inline-block" action="/model/<%= %>/stop" method="post">
    <button type="submit" class="btn btn-sm btn-danger">Stop training</button>
<% } else if (model.has_data) { %>
  <form class="d-inline-block" action="/model/<%= %>/start" method="post">
    <button type="submit" class="btn btn-sm btn-outline-primary">Start training</button>
<% } %>

Before executing training script we need to be sure it is not running currently, this is achieved by checking if training_pid is present on model instance. It is also necessary to clean up any existing log entries if those exist because relationship prohibits having more than one log representation for training, in other words - one model one log. Then script will be started in a separate process and events coming from it will be both stored in the database and sent to websocket connection to be rendered in real time. Websocket will not be covered here as it is part of other article I mentioned above “Using Python scripts in Node.js server”'/:id/start', checkPathParamSet("id"), loadInstanceById(), asyncErrHandler.bind(null, async (req, res) => {

  let model = req.instance
  // if in progress then cancel
  if (model.training_pid) {
    return res.redirect(`${req.baseUrl}/${}`)

  const params = JSON.parse(model.train_params || "{}")
  // clear any existing logging
  await db.deleteLogEntries(

  // launch script process
  let subprocess
  try {
    subprocess = await trainModel(, params)
  } catch (err) {
    // cleanup if error
    await db.setModelTrainingStopped(
    return res.render('show', Object.assign(res.locals, {
      model: req.instance,
      error: util.inspect(err)

  // obtain a running websocket instance
  const wss =

  // store logs and also spit it out to websocket ro render them in real time
  let chunkPosition = 1
  subprocess.stdout.on('data', async (data) => {
    let logEntry = {
      chunk: data + "",
      position: chunkPosition
    await db.insertLogEntry(logEntry)
  subprocess.stderr.on('data', async (data) => {
    let logEntry = {
      chunk: `Error: ${data}`,
      position: chunkPosition
    await db.insertLogEntry(logEntry)

  // mark model as in progress
  await db.setModelTrainingStarted(,;

  // get back to page it was initiated on

Script is launched in trainModel() function which returns Promise. In addition to checks above within router handler trainModel() will double check stored training parameters and will then merge them with defaults. Training data and pid file existence will also be checked before spawning a new Python process.

function trainModel(submissionId, params) {

  // defaults
  let args = {
    num_seqs: 32,
    num_steps: 50,
    lstm_size: 128,
    num_layers: 2,
    use_embedding: false,
    embedding_size: 128,
    learning_rate: 0.001,
    train_keep_prob: 0.5,
    max_steps: 1000,
    save_every_n: 1000,
    log_every_n: 100,
    max_vocab: 3500

  return new Promise(function (resolve, reject) {
    if (!submissionId) reject("submissionId required");

    if (typeof params === "object") {
      let errors = chackTrainParams(params)
      if (errors) {
        return reject(errors)
      } else {
        Object.assign(args, params)

    const folderPath = path.join(UPLOADS_PATH, submissionId)
    const trainFilePath = path.join(folderPath, TRAIN_FILENAME)
    if (!fs.existsSync(trainFilePath))
      return reject("missing training data file")

    const trainPidPath = path.join(folderPath, TRAIN_PID_FILENAME)
    const modelDir = path.join(GENERATOR_PATH, MODEL_DIR, submissionId)
    // remove any existing checkpoints

    let spawnArgs = [
      path.join(GENERATOR_PATH, ''),
      "--input_file", trainFilePath, // utf8 encoded text file
      "--name", submissionId // name of the model
    Object.keys(args).forEach((k) => {
      if (k != null && args[k] != null) {
    // run script
    const subprocess = spawn('python', spawnArgs, {
      stdio: ['ignore', "pipe", "pipe"]
    // store pid file
    // cleanup in case of error
    subprocess.on("error", () => {
    subprocess.on("exit", () => {


Above will spawn Python process which will execute training script I mentioned in the beginning. Some variables are not clear as they are not defined in this excerpt but you could always look at the source code to see what values they hold

Generating sample

This will be similar as in training, first I need a “button” in UI:

<% if(model.is_complete){ %>
  <button id="sampleBtn" type="button" class="btn btn-sm btn-primary" 
      onclick="generateSample()">Generate sample
<% } %>

<div id="sample" class="my-3" style="display: none">
  <p class="font-weight-bold">Generated sample</p>
  <div id="sampleOutput" class="bg-dark text-light p-3 small"></div>

  function generateSample() {
    $("#sampleBtn").prop("disabled", true)

    $.get("/model/<%= %>/sample")
      .done(function (data) {
      .fail(function (data) {
        $("#sampleOutput").text(JSON.stringify(data) || "Error occurred")
      .always(function () {
        $("#sampleBtn").prop("disabled", false)

Instead of making a form post I’m using some ajax here with a help from jQuery. For anyone wondering why I did not use front end framework for this it is to keep things simpler.

Router will handle /sample endpoint which in turn checks if model finished training via its is_complete flag. It is also necessary to use same parameters as were used for training to make sure correct model representation is created before extracting a text sample.

router.get('/:id/sample', checkPathParamSet("id"), loadInstanceById(), asyncErrHandler.bind(null, async (req, res) => {
  let model = req.instance
  if (!model.is_complete) {
    res.status(400).send({error: "Not ready yet"})

  // use same params from training
  const trainingParams = JSON.parse(model.train_params)
  let args = [
  ].reduce((memo, key) => {
    if (trainingParams[key] != null)
      memo[key] = trainingParams[key]
    return memo;
  }, {})

  // TODO somebody add start_string and max_length from query params
  let subprocess
  try {
    subprocess = await sampleModel(, args)
  } catch (err) {
    return res.status(400).send({error: util.inspect(err)})

  subprocess.stderr.on('data', (data) => {
    console.log(`Error: ${data}`)
  res.set('Content-Type', 'text/plain');

As you see it is missing start_string and max_length arguments, this will be left for someone else to complete and add some input fields to UI, for now I’ve set some defaults for it to work.

sampleModel() function is similar to trainModel() so it is not necessary for me to repeat it over here you can check it out in source code.


Implementation here uses a mix of languages which is a bit of a challenge to make sure it runs across different machines. There are couple of issues I’d like to solve with Docker. Firstly correct dependencies have to be used for both Python scripts and Node.js server, then I’d like to run it all with one command, it all needs to be wrapped to be run in production environment. To solve dependency issue one could use virtualenv and npm, to run it all in one command developer could write a shell script but to run it all on production environment you need some orchestration. All mentioned issues can easily be resolved with Docker.

Choosing a base Docker image here is a challenge as I could not find the one with both Node.js and Python support, choose one or the other and install missing components. I chose Python image as a base one as it is a bit more important to make certain those scripts run in consistent environment and I could not guarantee it would always be the same if it was installed from scratch at every deployment to production. Installing Node.js on the other hand is not hard at all when using nvm besides developer usually locks its dependencies with package-lock.json. Furthermore if anything goes wrong with server it will be easier to spot rather than checking in logs if scripts are failing.

Directory structure is split into server and generator. Former contains all Node.js server and latter the scripts.

  +- generator/
  +- server/
  \ Dockerfile

Dockerfile is simple enough without the part that installs Node.js

FROM python:3.6

# replace shell with bash so we can source files
RUN rm /bin/sh && ln -s /bin/bash /bin/sh

# Install node.js
ENV NVM_DIR /usr/local/nvm
RUN mkdir -p $NVM_DIR
# install nvm
RUN curl -o- | bash
# install node and npm
RUN echo "source $NVM_DIR/ && \
    nvm install $NODE_VERSION && \
    nvm alias default $NODE_VERSION && \
    nvm use default" | bash
# add node and npm to path so the commands are available
ENV NODE_PATH $NVM_DIR/versions/node/v$NODE_VERSION/lib/node_modules
ENV PATH $NVM_DIR/versions/node/v$NODE_VERSION/bin:$PATH
# confirm installation
RUN node -v
RUN npm -v

# prepare workdir
ENV SERVER_PATH /app/server
ENV GENERATOR_PATH /app/generator

# separate installation of python deps and copying python assets
# to make sure docker caches installation step
COPY generator/requirements.txt $GENERATOR_PATH/requirements.txt
RUN pip install -r $GENERATOR_PATH/requirements.txt

# separate installation of npm modules
# to make sure docker caches installation step
COPY server/package.json $SERVER_PATH/package.json
COPY server/package-lock.json $SERVER_PATH/package-lock.json
RUN cd $SERVER_PATH && npm install




CMD ["npm", "start"]

If you are not very familiar with Docker then all it does is installs Node.js along with nvm, then copies over the files and installs Python dependencies for scripts:

COPY generator/requirements.txt $GENERATOR_PATH/requirements.txt
RUN pip install -r $GENERATOR_PATH/requirements.txt

and then Node.js server dependencies:

COPY server/package.json $SERVER_PATH/package.json
COPY server/package-lock.json $SERVER_PATH/package-lock.json
RUN cd $SERVER_PATH && npm install

To run the above you have to install Docker on your machine and have MySQL running with rnn_generator database created and schema applied to it:

  • build container image: docker build -t foobar .
  • run built image:
    docker run --rm -ti \
      -e "MYSQL_HOST=docker.for.mac.localhost" \
      -e "MYSQL_USER=root" \
      -e "MYSQL_PASSWORD=" \
      -e "MYSQL_DATABASE=rnn_generator" \
      -p 8080:8080 foobar

docker.for.mac.localhost is meant for OSX users, if you use something else try finding out the value in Docker documentation.

Deployment to AWS

All this exercise was not only about being able to run those scripts locally but also to try and deploy them to environment close to production. For this reason I went with what I usually work with which is AWS. I say close to production because I do not intend to spend money on this example implementation I just made, it will run given tiny resources.

I need couple of parts of infrastructure to make it all work:

  • database - RDS
  • Docker container registry - Elastic Container Registry (ECR)
  • Docker runner - Elastic Beanstalk (EB)

I could have chosen Elastic Container Service (ECS) to run the Docker image on but I got scared trying to use it as there were some many options compared to EB launch configuration.

Docker files can be run on EB but then it builds them at the time of deployment which might take really long time before instance becomes ready to respond to requests. It is much better to build the image on CI server and push it to registry before using it.

CI server

This whole example is hosted on Github and is quite easy to integrate with Travis CI which is also free when used with public repositories. I used it for building the Docker image and pushing it to ECR in AWS. CI configuration is relatively simple not taking into account the shell script I had to assemble for it to build and push image to AWS. It will run Docker build every time there is a new git commit pushed to Github and then will deploy but only on master branch.

sudo: required
language: python
- docker
  - DOCKER_REPO=ivarprudnikov/rnn-generator
  - AWS_ACCOUNT_ID=<redacted>
  - EB_REGION="eu-west-1"
  - EB_APP="<elastic beanstalk app name>"
  - EB_ENV="<elastic beanstalk environment name>";
  - S3_BUCKET="<s3 bucket name the zipped app will be uploaded to>"
  - secure: <encrypted secret>
  - secure: <encrypted access key>
- pip install awscli
- export PATH=$PATH:$HOME/.local/bin
- docker build -t $DOCKER_REPO .
  provider: script
  script: bash
    branch: master

Deployment script is an assembled version from those I found in the wild internets, it relies on installed awscli sdk to tag the build that was just made and then to push it to registry.

#!/bin/bash -e

TIMESTAMP=$(date '+%Y%m%d%H%M%S')

aws configure set default.region ${EB_REGION}

# Push image to ECR

$(aws ecr get-login --no-include-email)

# update latest version
docker push ${TARGET_IMAGE_LATEST}

# push new version

# ...

I do push 2 tags here, the latest one and the versioned one which will allow me to specify it when deploying to Elastic Beanstalk. Keep in mind that ECR has some limits on maximum amount of tags and images. Deployment to EB requires me to push zip field with just the in it which in tun tells what docker image to use:

  "AWSEBDockerrunVersion": "2",
  "volumes": [],
  "containerDefinitions": [
      "name": "generator",
      "image": "<TARGET_IMAGE>",
      "essential": true,
      "memoryReservation": 96,
      "portMappings": [
          "hostPort": 80,
          "containerPort": 8080
      "mountPoints": []

json file contains <TARGET_IMAGE> which is replaced in deployment script with a versioned image name:

# ...


# Deploy new version to Elasticbeanstalk

# Interpolate and also create backup .bak file

# Zip application
zip -r ${ZIP}

# Copy application version over to S3
aws s3 cp ${ZIP} s3://${S3_BUCKET}/${ZIP}

# Create a new application version with the zipped up Dockerrun file
aws elasticbeanstalk create-application-version --application-name ${EB_APP} \
    --version-label ${VERSION} --source-bundle S3Bucket=${S3_BUCKET},S3Key=${ZIP}

# Update the environment to use the new application version
aws elasticbeanstalk update-environment --environment-name ${EB_ENV} \
      --version-label ${VERSION}

Elastic beanstalk

There is part of configuration on CI server which pushes 2 things to AWS: freshly built Docker image and new EB application version. This is almost everything we need except setting up EB environment itself which proved to be a bit catchy and time consuming so I’ll outline some important bits I got caught on.

To start I had to set up AWS RDS MySQL database which in turn resides in a VPC. Then I had to enable Networking under EB application which requires enabling LoadBalancer, the former allows setting up VPC details which are necessary to allow your EC2 instance to be created in that same VPC to then access the database. Apart from VPC which is out of scope here I needed to make sure my application is using correct image and then is correctly mapped in LoadBalancer.

app file structure
Load balancer port mapping

When creating an app make sure to select Multi-container Docker running on 64bit Amazon Linux as previously mentioned is using "AWSEBDockerrunVersion": "2".

app file structure
AWS EB app and container versions


Right this is a long one and took me some time to build. Eventually I’ve made it available to public under but I have little confidence in it working for more than one user as resources allocated to that instance are miserable.

This exercise explored a naive way of running Tensorflow powered scripts via webapp. Even this basic functionality which was achieved required to use wide array of development techniques to make it successiful:

  • Node.js powered web server
  • Tensorflow powered recurrent neural net implementation
  • Websockets to see live progress in UI
  • Docker to make sure the application is running in predictable environment
  • CI server to automate deployments
  • Docker container registry to store images
  • Elastic Beanstalk to run Docker container images
  • RDS to store sequential data
  • Bootstrap framework to assemble UI in html

If you asked me why is there no React in this list I might do something stupid.


Current implementation is clunky and hardly scalable as both training script and webserver lives on the same box. Making sure that scripts run in something with GPU would be a great start, I would probably try AWS Lambda or one of their Machine learning products.

After splitting out Python scripts it would be much easier to concentrate on implementing sort of REST API so that website could use more ajax instead of old school forms.

UI could also have more pictures of unicorns to make sure MVP attracts first million in days and not years.

Thank you for reading.

Source code

Source code is available in Github repo ivarprudnikov/char-rnn-tensorflow

Older post

Using Python scripts in Node.js server

November 11, 2018
A while back ago I was learning/playing with machine/deep learning but all that was only in Python, in a Jupyter Notebook. Latter allows writing and running and executing script...
Continue reading
Newer post

Build Docker image on Travis CI and push it to AWS ECR

January 17, 2019
If you are using AWS it is relatively easy to create a private Docker registry and after pushing some images, reference them when launching ECS/EB instances.
Continue reading