REST API

From Training Material
Revision as of 11:22, 16 May 2023 by Lsokolowski (talk | contribs) (→‎Exercises)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search


title
REST API: pattern of information exchange
author
Patrick Mazzotta, Lukasz Sokolowski


REST API Training Course ⌘


A developer's primer on REST Application Programming Interface.

About Us⌘

500res-logo.png

  • Training & Consultancy Provider
  • Founded in 2005
  • Currently offering over 900 courses (remote & on-site)
  • Continuously expanding through franchising opportunities

About Me

Craedone.png

  • Consulting in IT & BI for 20 years
  • CEO & Co-Founder of Craedone
    • Analytics
    • Business Intelligence
    • Corporate Strategy/Red Team services
  • Partnered with NobleProg since 2013


Expectations⌘

  • customization vs. course standards
    • content
    • ugly slides (content > appearance)
  • REST - zooming in and out throughout the material
  • Dialogue - me vs. reading a book


Terms.png
(hint: I'm not a very good book!)

Vocabulary: The Primer's Primer⌘

Wat.jpg

REST = REpresentational State Transfer
REST is an architectural style consisting of a coordinated set of architectural constraints applied to components, connectors, and data elements, within a distributed hypermedia system. REST ignores the details of component implementation and protocol syntax in order to focus on the roles of components, the constraints upon their interaction with other components, and their interpretation of significant data elements.[1]
API = Application Programming Interface
specifies a software component in terms of its operations, their inputs and outputs and underlying types. Its main purpose is to define a set of functionalities that are independent of their respective implementation, allowing both definition and implementation to vary without compromising each other.[2]
SOA = Service Oriented Architecure
a software design and software architecture design pattern based on distinct pieces of software providing application functionality as services to other applications. This is known as service-orientation. It is independent of any vendor, product or technology[3]
  1. Wikipedia:Representational State Transfer, [1]
  2. Wikipedia:Application Programming Interface, [2]
  3. Wikipedia:Service-Oriented Architecture, [3]

Vocabulary: The Primer's Primer (con't)⌘

JSON = JavaScript Object Notation
a standard for oganizing non-relational information for applications. Used heavily in REST systems and most NoSQL storage engines (i.e. MongoDB & Elasticsearch). It's best described as a key-value alternative to XML. JSON is not a 1:1 representation of actual JavaScript objects.[1]
RAML = RESTful API Modeling Language
a YAML-based language for describing RESTful APIs.[2] It provides all the information necessary to describe RESTful or practically-RESTful APIs. Although designed with RESTful APIs in mind, RAML is capable of describing APIs that do not obey all constraints of REST.[2]
SOAP = Simple Object Access Protocol
originally an acronym for Simple Object Access protocol, is a protocol specification for exchanging structured information in the implementation of web services in computer networks. It uses XML Information Set for its message format, and relies on other application layer protocols, most notably Hypertext Transfer Protocol (HTTP) or Simple Mail Transfer Protocol (SMTP), for message negotiation and transmission.[3]

Soap.jpg

  1. Wikipedia:JSON, [4]
  2. Wikipedia:RAML, [5]
  3. Wikipedia:SOAP, [6]

REST⌘

So what's the big deal about REST services anyway?

Glass half full kind of day.JPG

  • Verby (POST/GET/PUT/DELETE)
    • ex. CRUD (create/read/update/delete)
  • HTTP-based
    • minimal impact to high-level architecture
    • avoids client-side demands

What REST Is Not⌘

Beware of the golden hammer rule (a.k.a. law of the instrument)! There are things REST doesn't do so well:
GoldenHammer.jpg

  • high-volume transfer of structured (i.e. "known") data
  • fast simple responses (i.e. video games)

Remember: REST is not a standard itself - it uses the HTTP standard in a structured way.

REST Services⌘

What REST does do exceptionally well?

Simplifies interfaces for non-linearly scaled architecture.

Example REST services

Mapping CRUD To HTTP⌘

CRUD = Create Read Update Delete -- the cornerstones of persistent storage

Operation SQL HTTP DDS
Create INSERT PUT / POST write
Read (Retrieve) SELECT GET read / take
Update (Modify) UPDATE PUT / PATCH write
Delete (Destroy) DELETE DELETE dispose

[1] Using SOA design principles, we can imagine how this quickly scales to service specific application needs. Essentially, every URI maps to a single interface which in turn corresponds to a specific duty or operation within the system

  1. Wikipedia:CRUD, [7]

Conventions API URL structure⌘

Web service APIs that adhere to the REST architectural constraints are called RESTful. HTTP based RESTful APIs are defined with these aspects:

  • base URI, such as http://example.com/resources/
  • an Internet media type for the data. This is often JSON but can be any other valid Internet media type (e.g. XML, Atom, microformats, images, etc.)
  • standard HTTP methods (e.g., GET, PUT, POST, or DELETE)
  • hypertext links to reference state
  • hypertext links to reference related resources

[1]

  1. Wikipedia:Representational State Transfer, [8]

RESTFUL HTTP Methods⌘

RESTful API HTTP methods
Resource GET PUT POST DELETE
Collection URI, such as http://example.com/resources List the URIs and perhaps other details of the collection's members. Replace the entire collection with another collection. Create a new entry in the collection. The new entry's URI is assigned automatically and is usually returned by the operation. Delete the entire collection.
Element URI, such as http://example.com/resources/item17 Retrieve a representation of the addressed member of the collection, expressed in an appropriate Internet media type. Replace the addressed member of the collection, or if it doesn't exist, create it. Not generally used. Treat the addressed member as a collection in its own right and create a new entry in it. Delete the addressed member of the collection.


Passing Parameters⌘

  • use the specs! (GET vs POST vs PUT)
  • generally speaking, you'll see most APIs expect URL parameters
  • Simple Twitter interface for counting instances of a URL:
    • https://api.twitter.com/2/tweets/20 (use your api key, if you already have any)
      • curl -X GET -H "Authorization: Bearer <BEARER TOKEN>" "https://api.twitter.com/2/tweets/20"
    • let's play with the url param (440322224407314432)
      • curl -X GET -H "Authorization: Bearer <BEARER TOKEN>" "https://api.twitter.com/2/tweets/440322224407314432?expansions=author_id
      • curl -X GET -H "Authorization: Bearer <BEARER TOKEN>" "https://api.twitter.com/2/tweets/440322224407314432?expansions=author_id,attachments.media_keys"

HTTP Response Codes⌘

200 OK
General status code. Most common code used to indicate success.
201 CREATED
Successful creation occurred (via either POST or PUT). Expect the Location header to contain a link to the newly-created resource (on POST). Response body content may or may not be present.
204 NO CONTENT
Indicates success but nothing is in the response body, often used for DELETE and UPDATE operations.
400 BAD REQUEST
General error when fulfilling the request would cause an invalid state. Domain validation errors, missing data, etc. are some examples.
401 UNAUTHORIZED
Error code response for missing or invalid authentication token.
403 FORBIDDEN
Error code for user not authorized to perform the operation or the resource is unavailable for some reason (e.g. time constraints, etc.).
404 NOT FOUND
Used when the requested resource is not found, whether it doesn't exist or if there was a 401 or 403 that, for security reasons, the service wants to mask.
405 METHOD NOT ALLOWED
Used to indicate that the requested URL exists, but the requested HTTP method is not applicable. For example, POST /users/12345 where the API doesn't support creation of resources this way (with a provided ID). The Allow HTTP header must be set when returning a 405 to indicate the HTTP methods that are supported. In the previous case, the header would look like "Allow: GET, PUT, DELETE"
409 CONFLICT
Whenever a resource conflict would be caused by fulfilling the request. Duplicate entries, such as trying to create two customers with the same information, and deleting root objects when cascade-delete is not supported are a couple of examples.
500 INTERNAL SERVER ERROR
The general catch-all error when the server-side throws an exception. Expect this only for errors that haven't been otherwise handled.

More status codes from http://www.restapitutorial.com/httpstatuscodes.html

Return Values⌘

  • best practices vs. standards
  • API's are free to make their own choices
    • look before you leap!
  • JSON, JSON-P, XML

Anything goes! (for HTTP...)

JSON - a universal format for the exchange of information⌘

JSON
JavaScript Object Notation is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It is based on a subset of the JavaScript Programming Language, Standard ECMA-262 3rd Edition - December 1999. JSON is a text format that is completely language independent but uses conventions that are familiar to programmers of the C-family of languages, including C, C++, C#, Java, JavaScript, Perl, Python, and many others. These properties make JSON an ideal data-interchange language.[1]
  • Thank's Crockford!
  • It's not JavaScript!
  • Specifies five basic data types
    • Number
    • String
    • Boolean
    • Array
    • Object (associative array)
    • null
  • quickly becoming the leading format for document-based / non-relational storage engines
  1. JSON.org, [9]

JSON - what does it look like?⌘

{
  "firstName": "John",
  "lastName": "Smith",
  "isAlive": true,
  "age": 25,
  "height_cm": 167.6,
  "address": {
    "streetAddress": "21 2nd Street",
    "city": "New York",
    "state": "NY",
    "postalCode": "10021-3100"
  },
  "phoneNumbers": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "office",
      "number": "646 555-4567"
    }
  ],
  "children": [],
  "spouse": null
}


JSONP & CORS⌘

  • not technically data structures
JSONP
CORS
Http Action Supports only GET Supports GET, POST, PUT & DELETE, etc
No. of requests Only one Http request per call

Two Http requests per call (Pre-flight request & actual request)

Security Not Secured.Possibility for CSRF (Cross site request forgery). Since the data exchange is in the form of java script function, malware scripts could be injected Highly secured.The developer has control in setting up the security level by configuring the URLs to be allowed for CORS request.
Response headers No access Have access
Http Error Handling Http Error handling not possible since there is no access to response headers Nicer error handling since CORS is built up on top of XmlHttpRequest object
Authentication NA Supports basic and windows authentication through HTTP headers
Data Size Huge form data cannot be handled as there is a restriction in query string length. (GET request) Huge form data could be handled easily since it supports POST
Complex Data Type Serializing and de-serializing complex data types is a nightmare for the developers Not a burden. Easy to handle.
A-Sync Behavior Since, there would be a little wait time to load a script, this will seem to like synchronous CORS could be implemented completely in an A-Sync manner

[1]

As a consequence, CORS isn't backwards compatible with older browsers.

  1. JSONP vs CORS[10]

Passing Parameters (part 2)⌘

Based on the Twitter documentation[1]:

  • Base URL is: https://api.twitter.com/1.1/
  • The endpoint for a search call is search/tweets.json

So combined you get a REST interface at https://api.twitter.com/1.1/search/tweets.json

  • to look for tweets that have the phrase "REST API"
    • q="REST%20API"

Put it all together:

 https://api.twitter.com/1.1/search/tweets.json?q="REST%20API"
  • With curl: curl -X GET -H "Authorization: Bearer <BEARER_TOKEN>" \ "https://api.twitter.com/labs/2/tweets/search?tweet.fields=created_at,author_id,lang&query=from%3Atwitterdev%20has%3Amedia"
  1. Twitter:REST, [11]

Getting Twitter Setup⌘

  1. Log in to http://twitter.com
  2. after authentication, go to apps.twitter.com
  3. create a new app for yourself - feel free to use filler data
    1. read/write access level
    2. generate new keys

A Twitter Search⌘

Start at:

https://publish.twitter.com/#

https://developer.twitter.com/en/docs/twitter-api/tools-and-libraries

https://developer.twitter.com/en/docs/tutorials/using-twurl

  • /statuses/home_timeline.json
    • exclude_replies=true
  • /friends/ids.json
    • screen_name
  • /statuses/retweets_of_me
    • count
    • trim_user
  • favorites/list
    • screen_name
    • user_id

More Simple APIs⌘

Examples of popular APIs⌘

Creating a REST API⌘

Madscientist.jpg
Crafting genius!

Gathering requirements and identify functionality⌘

Think about:

  • who is your target audience?
  • how mission-critical is your API for users?
    • are there different levels of dependency amongst users?
  • who's data are you managing?
  • how expansive will your API be?

Design and API documentation ⌘

Think about:

  • idempotent concerns
    • Is delete idempotent? (hint: it should be)
  • openness vs. security
  • demand and scaling
  • Storage!
  • availability
  • versioning

Do:

  • Take notes (i.e. do not stick to just REST or only one technology in one language)
    • jQuery
    • Java
  • Always think about SOA
    • “eat your own dog food”

Documentation Tools⌘

Swaggger
https://swagger.io/
I/O Docs
https://github.com/mashery/iodocs
apiary
http://apiary.io/
Docco
http://jashkenas.github.io/docco/
Dexy
https://www.dexy.it/
Doxygen
https://www.doxygen.nl/

Installing the required libraries ⌘

A simiple/non-comprehensive list

  • Java
    • Spring
    • JavaLite
  • NodeJS
    • Express
    • Restify
    • Koa
    • Hapi
  • PHP
    • Httpful
    • Simple-REST
    • Dave
    • Zend (go big or go home!)
  • Python
    • Flask
    • Eve
    • Django

Business Object - mapping the class ⌘

Mapping.jpg
There are a few ways to achieve mapping

  • connect REST interfaces directly to your server object interfaces
    • risky business, not advised
  • connect REST interfaces through a sanitation router that ends at object interfaces
  • re-implement and/or re-instantiate your objects in stand-alone server

Controller methods ⌘

  • Using OO design principles
  • Avoid super-massive routes – break it down
  • Execution varies from language to language

Representation of a business object as JSON ⌘

hands-on walk through

HTTP Headers - as metadata ⌘

Headers are used to define the operation and scope of operations for a transaction/transmission. Things that can go into the header:

  • credentials (Authorization parameter)
Authorization:
OAuth realm="http://sp.example.test/",
oauth_consumer_key="0685bd9184jfhq22",
 oauth_token="ad180jjd733klru7",
 oauth_signature_method="HMAC-SHA1",
oauth_signature="wOJIO9A2W5mFwDgiDvZbTSMK%2FPY%3D",
 oauth_timestamp="137131200",
oauth_nonce="4572616e48616d6d65724c61686176",
oauth_version="1.0"
  • content descriptions
Date: Tue, 15 Nov 1994 08:12:31 GMT
Content-Type: application/x-www-form-urlencoded
Content-MD5: Q2hlY2sgSW50ZWdyaXR5IQ==
Content-Length: 348
  • terms of response acceptance
Accept: text/plain
Accept-Charset: utf-8
Accept-Encoding: gzip, deflate

Testing the API using cURL ⌘

curl is a tool to transfer data from or to a server, using one of the supported protocols (DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP). The command is designed to work without user interaction.[1]

Theoretical example:

curl -i -H "Accept: application/json" -X POST -d "firstName=james" http://api.something.com/users/user
  • i = show response headers
  • H = pass request headers to the resource
  • X = pass a HTTP method name
  • d = pass in parameters enclosed in quotes; multiple parameters are separated by ‘&’

The above command posts the first name “james” to the users resource. Assuming the server creates a new user resource with first name of James, we also ask the server to return a JSON response containing the newly created resource.
Actual example:

curl -i -d "url=microsoft.com" http://urls.api.twitter.com/1/urls/count.json
  1. Manpagez: curl, [12]

Authentication mechanisms ⌘

  • OAuth 1 & 2
    • v2 doens't require browser-based token assignment (don't have to go to a browser to hit the API)
    • v2 doesn't require cryptography (HTTPS is sufficient)
    • v2 has less parsing/sorting/encoding to handle
    • v1 allowed for long access token lives, v2 allows for refreshing of tokens
    • v2 allows for distribution of requests server-side (i.e. an auth role and interface role within the same request)
    • v2 is far less standardized - large teams (Facebook, Google, etc.) have slightly personallized implementations of v2

In short - OAuth 2 has some very attractive benefits, but the lack of uniformity causes some inherit risks (security and rate of adoption).
Combo lock.jpg


Authentication mechanisms (Con't)⌘

http://oauthlib.readthedocs.org/en/latest/oauth_1_versus_oauth_2.html

Recap ⌘

  • JSON
  • SOA principles
  • REST in a nutshell
  • Q&A

Exercises

  1. Clients:
  2. Servers:
1. Prepare simple REST api on free web-hosting environment
1.1. Create free webhosting account here:
http://www.000webhost.com/
1.2. Upload 'ex1' folder into newly created webhosting
1.2.1. Use FileManager icon
1.3. Test your first api
1.3.1. In the webbrowser go to your new website:
http://ulca.comule.com/ex1/index.php?method=hello
1.3.2. Replace 'ulca' with name you chose in host
1.4. Use your favorite web inspector (ex. firebug) and look at request/response
1.5. Change response format from default 'html' to 'json' and 'xml'
1.6. Improve the server with 'post' and write client program to test it


2. Prepare more advanced REST api
2.1. Upload 'ex2' folder
2.2. Via 'phpMyAdmin' create MySQL database
2.2.1. Create table 'users', use query from file 'ex2/api.php'
2.2.2. Insert at least one row to table 'users'
2.3. Change all MySQL necessary credentials in file 'ex2/api.php'
2.4. Test your second api
2.4.1. In the web-browser go to your new website:
http://ulca.comule.com/ex2/api.php?rquest=users
2.5. Improve your REST server
2.5.1. Create method 'add_new_user'
2.5.2. Create method 'update_existing_user'
2.6. Build a simple rest client 
2.6.1. Ask 'ex2' server for all users names
2.6.2. Add new user
2.6.3. Delete existing user
2.6.4. Update existing user


3. Create REST server with these instructions
3.1. https://www.drupal.org/node/2424977
3.2. https://www.drupal.org/project/services_views
3.3. https://www.drupal.org/node/1795770


4. Make REST server from this
4.1. https://docs.angularjs.org/tutorial/step_13