RESTful Web Services in Java

  • Assignment 1 is a "mini project" warm-up, a small web app running on a Java RESTful web server
  • Here we cover basic web programming for that assignment
  • For illustration we here use a "micro project", a To Do App (click on the previous for the sources at GitHub).
  • (Note we are not going to cover JavaScript front-ends here, that is a self-study topic.)

Web architecture evolution

Web programming is a rapidly evolving space.

  • Old school (90's) web: server makes full static HTML pages, renders
  • 00's web: web server makes pages, includes some JavaScript for callbacks to server for dynamic update of portions of pages
  • 10's web: single-page application (SPA) - avoid page reload; a javascript app running in the browser is re-writing the fixed webpage (the "DOM") to affect changes.

We are doing the 10's version.

RESTful Web Services: the browser/server interface

  • The (http) web server performs all the necessarily central actions
  • The JavaScript in the browser
    1. renders the data pulled from the server and
    2. accepts user input which may prompt it to make a request to server

Since the whole Internet lies between the browser and the server, a precise interface specification is needed for communication.

Principle: program to interfaces, not implementations

  • REST is a very simple but effective web server protocol for browser/server communication
  • "REST" stands for something but think of the server "at rest" until a request comes in from the browser JavaScript.
  • REST is not an actual protocol, it is a convention; the protocol used is good old HTTP.
  • REST is NOT just for browsers, any app needing data can use REST to get/put that data.
    -- one reason why we are covering this is a RESTful server is useful for many projects, e.g. serving data to your Android/iOS app
Basic properties of REST
  • There is a precise request/response data interface format
  • These days the data format is nearly always JSON -- {"msg" : "Hi I am a JSON string!", "priority" : "Low"}; can in principle be any string format
  • It is stateless: Don't assume the server knows anything about your state

REST request types ("methods")

REST runs on the standard HTTP protocol, there are several common HTTP methods indicating request type the browser can send.
  • GET - get some data
  • POST - put up new data
  • PUT - update or overwrite data
  • DELETE - delete data
  • others as well - see the full list of HTTP methods.

Request/response general format

  • A given http server request
    • is initiated by opening a network socket to e.g. pl.cs.jhu.edu port 80, the default http server port
    • includes various header info
    • includes the request type (GET/POST/etc) at a URI (e.g. GET /oose/index.shtml HTTP/1.0)
    • PLUS an arbitrary string argument (the body, where we put our JSON data)
  • The server then returns a response which includes
    • various header info including the status code (200 is the HTTP code meaning all went well, 4XX for various error types, etc - you have seen plenty of those.)
    • an arbitrary string body; it may be the html for a webpage or JSON data for a RESTful server response.

REST body request and response strings

  • Can be any string, but JSON is today's standard
         {   "title" : "this is JSON format",
             "comment": "isn't it simple??",
             "lucky_number" : 20,
             "feeling_lucky" : true }    
                  
  • JSON is a subset of JavaScript (the above code defines a JavaScript object) thus its popularity in REST.

We will experiment with RESTful HTTP on the above To Do example app, using Postman app to directly send JSON back and forth. The postman/ directory in the Todo source tree contains a collection of postman examples used in lecture that you can directly import.

Resource/route/endpoint terminology

  • Terminology: a resource is a particular data item to be accessed by REST (e.g. to do entries).
  • Terminology: a route is a URL suffix to address the resource needed, e.g. /todos/5 to refer to the 5th todo object
  • Terminology: an endpoint is a route plus an HTTP method (GET/PUT/DELETE/...)

The endpoints for the todo app (i.e., its RESTful interface)

To be clear about the endpoints supported by our API, we need to write it out. Here is a first pass; we should also specifiy the failure codes to be complete.

Add a Todo
Method POST
URL /api/v1/todos
Body { title : <string>, done: <boolean>, createdOn: <date string> }
Success Response Code: 201 created
  Body: {}
Get Todo numbered :id
Method GET
URL /api/v1/todos/:id
Body (none)
Success Response Code: 200 OK
  Body: { title : <string>, done: <boolean>, createdOn: <date string> }
Get list of all Todos
Method GET
URL /api/v1/todos
Body (none)
Success Response Code: 200 OK
  Body: [<todo 1>, <todo 2>, ...]
Update Todo numbered :id
Method PUT
URL /api/v1/todos/:id
Body { title : <string>, done: <boolean>, createdOn: <date string> }
Success Response Code: 200 OK
  Body: { title : <string>, done: <boolean>, createdOn: <date string> }
Delete Todo numbered :id
Method DELETE
URL /api/v1/todos/:id
Body (none)
Success Response Code: 200 OK
  Body: {}

An excellent industry example of endpoint documentation: Spotify

Under the hood of the server

  • RESTful GET/POST/ETC requests come in to the web server and need to be dispatched/routed to Java code to run - how?
  • We use a very simple server, SparkJava - endpoints are set up in Java code itself

SparkJava

  • Inspired by Ruby's Sinatra and also similar to Python's Flask
  • Part of a new industry "Microservices" trend to use multiple lightweight RESTful servers
  • We are not necessarily advocating for the microservices approach overall, but they are recommended for your projects since they are much simpler/understandable than the old heavyweights (J2EE, Tomcat, Django, Rails, etc).
  • SparkJava is just a Java package so no need to install anything, just add the Maven dependency
  • When you run your app that uses SparkJava it will fire up the internal web server automatically for you
  • The SparkJava documentation shows how simple it is - just one webpage!

SparkJava Todo app classes

  • Bootstrap - the main class
  • Todo - a "model" class, the regular Java class ("POJO") for a single To Do entry is the only model class here
  • TodoController - "controller" means dealing with input; this key class dispatches the different routes/endpoints
  • TodoService - front-end to the database, implements add/remove/etc
    -- could have had this code in Controller but remember separation of concerns!
  • JsonTransformer - convert Todo objects to and from JSON format
  • TodoController's setupEndpoints() method sets up all the routes
    -- specify what action to do when a particular route is requested

We will look at this setupEndpoints method in detail. Static methods spark.Spark.get(...) in setupEndpoints() add routes to the routing table, using Java 8 higher-order functions to "plug in" the route handling code.

The Java Model on the server: Todo class

  • The core of the server is regular Java classes (POJO's) for modeling the domain at hand
  • The Java model code should not itself be concerned with persisting data or making JSON for the browser
    -- keep a clear separation of concerns
  • The Todo class is the only model class in this simple example; in a larger app there will be many
    -- also, any underyling operations on this data should be methods on the model classes

See the Todo class for the Java model for the To Do list - its very simple code since its an incredibly simple app. Note that the Todo class only really has getters and setters, no methods; if your data has non-trivial operations on it they should be methods on the POJO.

The Persistence layer

This term usually refers to the database on the back-end - nearly all web services are serving persistent data from some kind of database.

  • We are not covering databases in detail here, but basic database use is not hard
  • The Todo example uses SQLite, a very lightweight database which works great for small projects
  • Postgresql and others are more industrial-scale
  • "NoSql" databases don't store databases in tables - a relatively recent trend

If you run the Todo app, a file todo.db is created which contains the SQLite database. We will inspect that file in class using a database browser.

Mapping between three data representations: JSON - Java object - database row

  • We have created a little problem, there are three different representations of the same data that need to be connected
  • This is a classic issue in web frameworks
  • To divide and conquer this problem, make two independent mappings
    • JSON - Java object
    • Java object - database
  • You might be tempted to short-circuit all these representations, but even though its a bit of a pain it separates concerns very well and leads to more robust code -- make sure you do this in your projects!

In the Todo app the TodoService class implements the controller tasks and oversees the mappings in both directions.

The JSON - Java object bidirectional mapping

  • Just do the data conversion - map JSON field -- Java object field
  • There are fortunately libraries to help, e.g. Google's Gson can automatically map field -- field in both directions.

The Todo app makes several uses of Gson to transform - see classes JsonTransformer and TodoService/TodoController

The Java object - database bidirectional mapping

There are two schools of thought here
  • "ORM" (Object relational mapping)
    • This generally means there is some framework doing an automatic conversion between Java objects and database table rows.
    • Hibernate is most widely used Java ORM
    • Its a lot of work to set up but then things work "magically"
    • But, when the magic breaks its hard for the non-expert to fix it
  • "POJO" (Plain old Java Object)
    • Don't use a mapping framework, just map manually.
    • This used to be hard but new libraries make it relatively easier
    • We will be using the sql2o package to map, it is more useful than the Java JDBC mapping
    • We will need to manually invoke SQL queries to read from DB, its a little bit but not too much pain.

See class TodoService for examples of sql2o being used to query the database and get POJOs as a result - db and conn here are database and connection objects that have sql2o classes.

Observe that sql2o uses method chaining to allow for compact construction of database queries by a series of message sends.

Here is a crude picture summarizing the above that I drew in class:

The Presentation layer aka the browser

  • Browsers used to be passive renderers of HTML files, no programs could be run in an out-of-the box browser
  • JavaScript changed that. It started as a small scripting language and grew into a "real" PL.
  • The key for us is the XMLHttpRequest API added to all leading browsers in the early 00's
  • XMLHttpRequest lets JavaScript code itself make HTTP GET/POST/etc requests
  • Another cool API added to all browsers in the early 00's is the DOM (Document Object model) manipulation API
    • the DOM is the tree representing the HTML of the webpage
    • the JavaScript DOM API in the browser lets JavaScript programs manipulate the DOM
    • When the program manipulates the DOM, the HTML changes, and the browser (automatically) renders the updated HTML
    • which means a whole website can really just be one initial webpage ("SPA") which is continually mutated by DOM updates.
  • People realized they could use XMLHttpRequest plus DOM to create SPA webpages, and they called it .. AJAX.

JavaScript APIs for AJAX programming

  • There are countless frameworks available; we recommend using one of the big three: React, Angular, or Ember.
  • We are not going to lecture on or require any JavaScript programming in this class, but its not so hard to learn on your own for your project

If there is time we will briefly inspect the JavaScript sources for the above To Do example app, file ../src/main/resources/scripts/app.js, to give you a rough idea of how to use Angular to write an app running in the browser.

Other aspects of Web programming

This has been a simplified look at web programming, many topics were skipped.

  • Security - a critically important issue; it is apparent that the Todo app has no data security once someone has access to the port it is running on.
  • Push - the server cannot push any data to the webpage under the simple model described; instead the javascript must periodically poll for updates. Web sockets are the answer to this - the 20's web.
  • Session management via cookies
  • Template engines (JSP/Velocity/Handlebars, etc) for rendering pages server-side (00's web, still used as a component of 10's web)

Appendix: Maven

Maven is the modern way to build a Java project. Gradle is similar to Maven, Android uses Gradle.

  • Like a makefile's targets, you can define Maven phases for different lifecycle functions in the Maven pom.xml configuration file
    • clean
    • validate - make sure you have all required packages etc.
    • compile
    • test
    • package - put all into a standalone Jar file
    • install
    • site - generate documentation
    • deploy
  • You can easily declare dependencies, packages you depend on
    - and the reaallllly cool part is all you have to do is put in the name/version in the pom.xml and Maven will download the jars for you
  • Maven plugins are extensions with a similarly easy auto-download method
  • Key to projects: share a Maven build file and your team will automatically be using identical versions of libraries etc - minimize platform-specific bugs.
  • We are going to require a proper pom.xml file for projects and Assignment 1 with all dependencies declared

We will inspect the To Do example app pom.xml file and build some of the other lifecycle phases

See the Tools page Maven entry for more information.