Prerequisites

• MongoDB
• Node.js 8.0+
• Redis
• ffmpeg
• bower

Please also check the Wiki for additional information on getting site functionality running

• Command Line Tools
•  Mac OS X: Xcode (or OS X 10.9+: xcode-select --install)
•  Windows: Visual Studio
•  Ubuntu /  Linux Mint: sudo apt-get install build-essential
•  Fedora: sudo dnf groupinstall "Development Tools"
•  OpenSUSE: sudo zypper install --type pattern devel_basis

Note: If you are new to Node or Express, I recommend to watch Node.js and Express 101 screencast by Alex Ford that teaches Node and Express from scratch. Alternatively, here is another great tutorial for complete beginners - Getting Started With Node.js, Express, MongoDB.

Getting Started

The easiest way to get started is to clone the repository:

# Get the latest snapshot
git clone https://github.com/mayeaux/nodetube

# Change directory
cd nodetube

# Install backend and frontend dependencies
npm run installDeps

# Then simply start your app
npm start

#If you're developing locally, you can boot the app with nodemon with:
npm run dev

Note: I highly recommend installing Nodemon. It watches for any changes in your node.js app and automatically restarts the server. Once installed, instead of node app.js use nodemon app.js. It will save you a lot of time in the long run, because you won't need to manually restart the server each time you make a small change in code. To install, run sudo npm install -g nodemon.

Yarn vs NPM

Yarn is a new JavaScript package manager built by Facebook, Google, Exponent and Tilde. Yarn is not an attempt to replace npm, it's simply an alternative CLI client for fetching modules from the npm registry but it does have some unique benefits over using npm, most noticeably speed and consistency (via a lock file which ensures that only specific versions of dependencies are installed). Hackathon Starter includes a yarn.lock file by default and as project dependencies are updated, this file will be updated to reflect those changes.

To upgrade your local dependencies using Yarn, simply run yarn upgrade. This will update all dependencies to their latest version based on the version range specified in the package.json file. The yarn.lock file will be recreated as well. For further information, please see the official documention for managing dependencies and upgrading dependencies. This Yarn vs NPM article by SitePoint also has some very useful information.

Docker

The project has an experimental Docker setup, using Docker Compose. It is entirely optional, so if you don't want to use Docker, feel free to ignore it.

If you have Docker installed, there's a non-zero chance that running docker-compose up is all you need to do to build and start 3 docker containers (one each for the app itself, Redis, and MongoDB). From there, you should be able to connect to http://localhost:49160 and see the app running.

Project Structure

Name	Description
config/passport.js	Passport Local and OAuth strategies, plus login middleware.
controllers/api.js	Controller for /api route and all api examples.
controllers/contact.js	Controller for contact form.
controllers/home.js	Controller for home page (index).
controllers/user.js	Controller for user account management.
models/User.js	Mongoose schema and model for User.
public/	Static assets (fonts, css, js, img).
public/js/application.js	Specify client-side JavaScript dependencies.
public/js/main.js	Place your client-side JavaScript here.
public/css/main.scss	Main stylesheet for your app.
public/css/themes/default.scss	Some Bootstrap overrides to make it look prettier.
views/account/	Templates for login, password reset, signup, profile.
views/api/	Templates for API Examples.
views/partials/flash.pug	Error, info and success flash notifications.
views/partials/header.pug	Navbar partial template.
views/partials/footer.pug	Footer partial template.
views/layout.pug	Base template.
views/home.pug	Home page template.
.env.example	Your API keys, tokens, passwords and database URI.
app.js	The main application file.
package.json	NPM dependencies.
yarn.lock	Contains exact versions of NPM dependencies in package.json.

Note: There is no preference how you name or structure your views. You could place all your templates in a top-level views directory without having a nested folder structure, if that makes things easier for you. Just don't forget to update extends ../layout and corresponding res.render() paths in controllers.

List of Packages

Package	Description
async	Utility library that provides asynchronous control flow.
bcrypt-nodejs	Library for hashing and salting user passwords.
cheerio	Scrape web pages using jQuery-style syntax.
clockwork	Clockwork SMS API library.
connect-mongo	MongoDB session store for Express.
dotenv	Loads environment variables from .env file.
express	Node.js web framework.
body-parser	Express 4 middleware.
express-session	Express 4 middleware.
morgan	Express 4 middleware.
compression	Express 4 middleware.
errorhandler	Express 4 middleware.
serve-favicon	Express 4 middleware offering favicon serving and caching.
express-flash	Provides flash messages for Express.
express-status-monitor	Reports real-time server metrics for Express.
express-validator	Easy form validation for Express.
fbgraph	Facebook Graph API library.
github	GitHub API library.
pug (jade)	Template engine for Express.
lastfm	Last.fm API library.
instagram-node	Instagram API library.
lob	Lob API library
lusca	CSRF middleware.
mongoose	MongoDB ODM.
node-foursquare	Foursquare API library.
node-linkedin	LinkedIn API library.
node-sass-middleware	Sass middleware compiler.
nodemailer	Node.js library for sending emails.
passport	Simple and elegant authentication library for node.js
passport-facebook	Sign-in with Facebook plugin.
passport-github	Sign-in with GitHub plugin.
passport-google-oauth	Sign-in with Google plugin.
passport-twitter	Sign-in with Twitter plugin.
passport-instagram	Sign-in with Instagram plugin.
passport-local	Sign-in with Username and Password plugin.
passport-linkedin-oauth2	Sign-in with LinkedIn plugin.
passport-oauth	Allows you to set up your own OAuth 1.0a and OAuth 2.0 strategies.
paypal-rest-sdk	PayPal APIs library.
request	Simplified HTTP request library.
stripe	Offical Stripe API library.
tumblr.js	Tumblr API library.
twilio	Twilio API library.
twit	Twitter API library.
lodash	Handy JavaScript utlities library.
validator	Used in conjunction with express-validator in controllers/api.js.
mocha	Test framework.
chai	BDD/TDD assertion library.
supertest	HTTP assertion library.

FAQ

Why do I get 403 Error: Forbidden when submitting a form?

You need to add the following hidden input element to your form. This has been added in the pull request #40 as part of the CSRF protection.

input(type='hidden', name='_csrf', value=_csrf)

Note: It is now possible to whitelist certain URLs. In other words you can specify a list of routes that should bypass CSRF verification check.

Note 2: To whitelist dynamic URLs use regular expression tests inside the CSRF middleware to see if req.originalUrl matches your desired pattern.

I am getting MongoDB Connection Error, how do I fix it?

That's a custom error message defined in app.js to indicate that there was a problem connecting to MongoDB:

mongoose.connection.on('error', () => {
  console.error('MongoDB Connection Error. Please make sure MongoDB is running.');
});

You need to have a MongoDB server running before launching app.js. You can download MongoDB here, or install it via a package manager. Windows users, read Install MongoDB on Windows.

Tip: If you are always connected to the internet, you could just use mLab or Compose instead of downloading and installing MongoDB locally. You will only need to update database credentials in .env file.

I get an error when I deploy my app, why?

Chances are you haven't changed the Database URI in .env. If MONGODB/MONGOLAB_URI is set to localhost, it will only work on your machine as long as MongoDB is running. When you deploy to Heroku, OpenShift or some other provider, you will not have MongoDB running on localhost. You need to create an account with mLab or Compose, then create a free tier database. See Deployment for more information on how to setup an account and a new database step-by-step with mLab.

Why Pug (Jade) instead of Handlebars?

When I first started this project I didn't have any experience with Handlebars. Since then I have worked on Ember.js apps and got myself familiar with the Handlebars syntax. While it is true Handlebars is easier, because it looks like good old HTML, I have no regrets picking Jade over Handlebars. First off, it's the default template engine in Express, so someone who has built Express apps in the past already knows it. Secondly, I find extends and block to be indispensable, which as far as I know, Handlebars does not have out of the box. And lastly, subjectively speaking, Jade looks much cleaner and shorter than Handlebars, or any non-HAML style for that matter.

How do flash messages work in this project?

Flash messages allow you to display a message at the end of the request and access it on next request and only next request. For instance, on a failed login attempt, you would display an alert with some error message, but as soon as you refresh that page or visit a different page and come back to the login page, that error message will be gone. It is only displayed once. This project uses express-flash module for flash messages. And that module is built on top of connect-flash, which is what I used in this project initially. With express-flash you don't have to explicitly send a flash message to every view inside res.render(). All flash messages are available in your views via messages object by default, thanks to express-flash.

Flash messages have a two-step process. You use req.flash('errors', { msg: 'Error messages goes here' } to create a flash message in your controllers, and then display them in your views:

if messages.errors
  .alert.alert-danger.fade.in
    for error in messages.errors
      div= error.msg

In the first step, 'errors' is the name of a flash message, which should match the name of the property on messages object in your views. You place alert messages inside if message.errors because you don't want to show them flash messages are actually present. The reason why you pass an error like { msg: 'Error messages goes here' } instead of just a string - 'Error messages goes here', is for the sake of consistency. To clarify that, express-validator module which is used for validating and sanitizing user's input, returns all errors as an array of objects, where each object has a msg property with a message why an error has occurred. Here is a more general example of what express-validator returns when there are errors present:

[
  { param: "name", msg: "Name is required", value: "<received input>" },
  { param: "email", msg: "A valid email is required", value: "<received input>" }
]

To keep consistent with that style, you should pass all flash messages as { msg: 'My flash message' } instead of a string. Otherwise you will just see an alert box without an error message. That is because, in partials/flash.pug template it will try to output error.msg (i.e. "My flash message".msg), in other words it will try to call a msg method on a String object, which will return undefined. Everything I just mentioned about errors, also applies to "info" and "success" flash messages, and you could even create a new one yourself, such as:

Data Usage Controller (Example)

req.flash('warning', { msg: 'You have exceeded 90% of your data usage' });

User Account Page (Example)

if messages.warning
  .alert.alert-warning.fade.in
    for warning in messages.warning
      div= warning.msg

partials/flash.pug is a partial template that contains how flash messages are formatted. Previously, flash messages were scattered throughout each view that used flash messages (contact, login, signup, profile), but now, thankfully it is uses a DRY approach.

The flash messages partial template is included in the layout.pug, along with footer and navigation.

body
    include partials/header

    .container
      include partials/flash
      block content

    include partials/footer

If you have any further questions about flash messages, please feel free to open an issue and I will update this mini-guide accordingly, or send a pull request if you would like to include something that I missed.

---

How do I create a new page?

A more correct way to be to say "How do I create a new route". The main file app.js contains all the routes. Each route has a callback function associated with it. Sometimes you will see 3 or more arguments to routes. In cases like that, the first argument is still a URL string, while middle arguments are what's called middleware. Think of middleware as a door. If this door prevents you from continuing forward, you won't get to your callback function. One such example is a route that requires authentication.

app.get('/account', passportConfig.isAuthenticated, userController.getAccount);

It always goes from left to right. A user visits /account page. Then isAuthenticated middleware checks if you are authenticated:

exports.isAuthenticated = (req, res, next) => {
  if (req.isAuthenticated()) {
    return next();
  }
  res.redirect('/login');
};

If you are authenticated, you let this visitor pass through your "door" by calling return next();. It then proceeds to the next middleware until it reaches the last argument, which is a callback function that typically renders a template on GET requests or redirects on POST requests. In this case, if you are authenticated, you will be redirected to Account Management page, otherwise you will be redirected to Login page.

exports.getAccount = (req, res) => {
  res.render('account/profile', {
    title: 'Account Management'
  });
};

Express.js has app.get, app.post, app.put, app.delete, but for the most part you will only use the first two HTTP verbs, unless you are building a RESTful API. If you just want to display a page, then use GET, if you are submitting a form, sending a file then use POST.

Here is a typical workflow for adding new routes to your application. Let's say we are building a page that lists all books from database.

Step 1. Start by defining a route.

app.get('/books', bookController.getBooks);

---

Note: As of Express 4.x you can define you routes like so:

app.route('/books')
  .get(bookController.getBooks)
  .post(bookController.createBooks)
  .put(bookController.updateBooks)
  .delete(bookController.deleteBooks)

And here is how a route would look if it required an authentication and an authorization middleware:

app.route('/api/twitter')
  .all(passportConfig.isAuthenticated)
  .all(passportConfig.isAuthorized)
  .get(apiController.getTwitter)
  .post(apiController.postTwitter)

Use whichever style that makes sense to you. Either one is acceptable. I really think that chaining HTTP verbs on app.route is very clean and elegant approach, but on the other hand I can no longer see all my routes at a glance when you have one route per line.

Step 2. Create a new schema and a model Book.js inside the models directory.

const mongoose = require('mongoose');

const bookSchema = new mongoose.Schema({
  name: String
});

const Book = mongoose.model('Book', bookSchema);
module.exports = Book;

Step 3. Create a new controller file called book.js inside the controllers directory.

/**
 * GET /books
 * List all books.
 */
const Book = require('../models/Book.js');

exports.getBooks = (req, res) => {
  Book.find((err, docs) => {
    res.render('books', { books: docs });
  });
};

Step 4. Import that controller in app.js.

const bookController = require('./controllers/book');

Step 5. Create books.pug template.

extends layout

block content
  .page-header
    h3 All Books

  ul
    for book in books
      li= book.name

That's it! I will say that you could have combined Step 1, 2, 3 as following:

app.get('/books',(req, res) => {
  Book.find((err, docs) => {
    res.render('books', { books: docs });
  });
});

Sure, it's simpler, but as soon as you pass 1000 lines of code in app.js it becomes a little difficult to navigate the file. I mean, the whole point of this boilerplate project was to separate concerns, so you could work with your teammates without running into MERGE CONFLICTS. Imagine you have 4 developers working on a single app.js, I promise you it won't be fun resolving merge conflicts all the time. If you are the only developer then it's fine. But as I said, once it gets up to a certain LoC size, it becomes difficult to maintain everything in a single file.

That's all there is to it. Express.js is super simple to use. Most of the time you will be dealing with other APIs to do the real work: Mongoose for querying database, socket.io for sending and receiving messages over websockets, sending emails via Nodemailer, form validation using express-validator library, parsing websites using Cheerio, and etc.

---

Deployment

Once you are ready to deploy your app, you will need to create an account with a cloud platform to host it. These are not the only choices, but they are my top picks. From my experience, Heroku is the easiest to get started with, it will automatically restart your Node.js process when it crashes, zero-downtime deployments and custom domain support on free accounts. Additionally, you can create an account with mLab and then pick one of the 4 providers below. Again, there are plenty of other choices and you are not limited to just the ones listed below.

1-Step Deployment with Heroku

- Download and install [Heroku Toolbelt](https://toolbelt.heroku.com/) - In terminal, run `heroku login` and enter your Heroku credentials - From *your app* directory run `heroku create` - Run `heroku addons:create mongolab`. This will set up the mLab add-on and configure the `MONGOLAB_URI` environment variable in your Heroku app for you. - Lastly, do `git push heroku master`. Done!

Note: To install Heroku add-ons your account must be verified.

---

- Open [mlab.com](https://mlab.com) website - Click the yellow **Sign up** button - Fill in your user information then hit **Create account** - From the dashboard, click on **:zap:Create new** button - Select **any** cloud provider (I usually go with AWS) - Under *Plan* click on **Single-node (development)** tab and select **Sandbox** (it's free) - *Leave MongoDB version as is - `2.4.x`* - Enter *Database name** for your web app - Then click on **:zap:Create new MongoDB deployment** button - Now, to access your database you need to create a DB user - Click to the recently created database - You should see the following message: - *A database user is required to connect to this database.* **Click here** *to create a new one.* - Click the link and fill in **DB Username** and **DB Password** fields - Finally, in `.env` instead of `mongodb://localhost:27017/test`, use the following URI with your credentials: - `db: 'mongodb://USERNAME:PASSWORD@ds027479.mongolab.com:27479/DATABASE_NAME'`

Note: As an alternative to mLab, there is also Compose.

- First, install this Ruby gem: `sudo gem install rhc` 💎 - Run `rhc login` and enter your OpenShift credentials - From your app directory run `rhc app create MyApp nodejs-0.10` - **Note:** *MyApp* is the name of your app (no spaces) - Once that is done, you will be provided with **URL**, **SSH** and **Git Remote** links - Visit provided **URL** and you should see the *Welcome to your Node.js application on OpenShift* page - Copy and and paste **Git Remote** into `git remote add openshift YOUR_GIT_REMOTE` - Before you push your app, you need to do a few modifications to your code

Add these two lines to app.js, just place them anywhere before app.listen():

var IP_ADDRESS = process.env.OPENSHIFT_NODEJS_IP || '127.0.0.1';
var PORT = process.env.OPENSHIFT_NODEJS_PORT || 8080;

Then change app.listen() to:

app.listen(PORT, IP_ADDRESS,() => {
  console.log(`Express server listening on port ${PORT} in ${app.settings.env} mode`);
});

Add this to package.json, after name and version. This is necessary because, by default, OpenShift looks for server.js file. And by specifying supervisor app.js it will automatically restart the server when node.js process crashes.

"main": "app.js",
"scripts": {
  "start": "supervisor app.js"
},

• Finally, you can now push your code to OpenShift by running git push -f openshift master
• Note: The first time you run this command, you have to pass -f (force) flag because OpenShift creates a dummy server with the welcome page when you create a new Node.js app. Passing -f flag will override everything with your Hackathon Starter project repository. Do not run git pull as it will create unnecessary merge conflicts.
• And you are done!

• Login to Windows Azure Management Portal
• Click the + NEW button on the bottom left of the portal
• Click COMPUTE, then WEB APP, then QUICK CREATE
• Enter a name for URL and select the datacenter REGION for your web site
• Click on CREATE WEB APP button
• Once the web site status changes to Running, click on the name of the web site to access the Dashboard
• At the bottom right of the Quickstart page, select Set up a deployment from source control
• Select Local Git repository from the list, and then click the arrow
• To enable Git publishing, Azure will ask you to create a user name and password
• Once the Git repository is ready, you will be presented with a GIT URL
• Inside your Hackathon Starter directory, run git remote add azure [Azure Git URL]
• To push your changes simply run git push azure master
• Note: You will be prompted for the password you created earlier
• On Deployments tab of your Windows Azure Web App, you will see the deployment history

---

IBM Bluemix Cloud Platform

1. Create a Bluemix Account

   Sign up for Bluemix, or use an existing account.

2. Download and install the Cloud Foundry CLI to push your applications to Bluemix.

3. Create a manifest.yml file in the root of your application.

applications:
- name:      <your-app-name>
  host:      <your-app-host>
  memory:    128M
  services:
  - myMongo-db-name

The host you use will determinate your application url initially, e.g. <host>.mybluemix.net.
The service name 'myMongo-db-name' is a declaration of your MongoDB service. If you are using other services like Watson for example, then you would declare them the same way.

1. Connect and login to Bluemix via the Cloud-foundry CLI

$ cf login -a https://api.ng.bluemix.net

1. Create a MongoDB service

$ cf create-service mongodb 100 [your-service-name]

Note: this is a free and experiment verion of MongoDB instance.
Use the MongoDB by Compose instance for production applications:

$ cf create-service compose-for-mongodb Standard [your-service-name]'

1. Push the application

   $ cf push
   

   $ cf env <your-app-name >
   (To view the *environment variables* created for your application)
   
   

Done, now go to the staging domain(<host>.mybluemix.net.) and see your app running.

Cloud Foundry Commands
More Bluemix samples
Simple ToDo app in a programming language of your choice

IBM Watson

Be sure to check out the full list of Watson services to forwarder enhance your application functionality with a little effort. Watson services are easy to get going, it is simply an RESTful API call. Here is an example of a Watson Toner Analyzer to understand the emotional context of a piece of text that you send to Watson.

Watson catalog of services

AlchemyAPI - An AlchemyAPI service that analyzes your unstructured text and image content.

Cognitive Commerce - Cognitive Commerce is a service provided by Cognitive Scale.

Cognitive Graph - Cognitive Graph is a service provided by Cognitive Scale.

Cognitive Insights - Cognitive Insights™ is a service provided by Cognitive Scale.

Conversation - Add a natural language interface to your application to automate interactions with your end users. Common applications include virtual agents and chat bots that can integrate and communicate on any channel or device.

Discovery - Add a cognitive search and content analytics engine to applications.

Document Conversion - Converts a HTML, PDF, or Microsoft Word™ document into a normalized HTML, plain text, or a set of JSON-formatted Answer units.

Language Translator - Translate text from one language to another for specific domains.

Natural Language Classifier - Natural Language Classifier performs natural language classification on question texts. A user would be able to train their data and the predict the appropriate class for a input question.

Personality Insights - The Watson Personality Insights derives insights from transactional and social media data to identify psychological traits.

Retrieve and Rank - Add machine learning enhanced search capabilities to your application.

Speech to Text - Low-latency, streaming transcription.

Text to Speech - Synthesizes natural-sounding speech from text.

Tone Analyzer - Tone Analyzer uses linguistic analysis to detect three types of tones from communications: emotion, social, and language. This insight can then be used to drive high impact communications.

Tradeoff Analytics - Helps make better choices under multiple conflicting goals. Combines smart visualization and recommendations for tradeoff exploration.

Visual Recognition - Find meaning in visual content! Analyze images for scenes, objects, faces, and other content. Choose a default model off the shelf, or create your own custom classifier. Find similar images within a collection. Develop smart applications that analyze the visual content of images or video frames to understand what is happening in a scene.

Click here for live demos of each Watson service.

---

Google Cloud Platform

• Download and install Node.js

• Select or create a Google Cloud Platform Console project

• Enable billing for your project (there's a $300 free trial)

• Install and initialize the Google Cloud SDK

• Create an app.yaml file at the root of your hackathon-starter folder with the following contents:

  runtime: nodejs
  vm: true
  manual_scaling:
    instances: 1

• Make sure you've set MONGODB_URI or MONGOLAB_URI in .env.example

• Run the following command to deploy the hackathon-starter app:

  gcloud app deploy

• Monitor your deployed app in the Cloud Console

• View the logs for your app in the Cloud Console

License

The MIT License (MIT)

Copyright (c) 2014-2016 Anthony Mayfield

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.