Symfony a RESTFul app: Security ( Securing the token path )

In the previous post we set up a basic symfony project aiming to develop a RESTFul solution, we also talk about the security and how to implement an OAuth2 service, but we also saw a security flaw in one of the most important end points, the /oauth/v2/token one which will deliver the access token to the user.

As always all the code on this series can be found on this github repository under the symfony2_restful_example branch.

Rest Authentication:

As we discuss earlier, the main concern is that the token end point suppose to receive the username and the password of the user/customer in plain text, also the end point is not protected with any authentication mechanism. In my personal opinion this is not a good idea, so what do we did to fix this security issue. The main idea goes with this points:

  • Implement a new authentication mechanism that receive the username and the hashed password of the user/customer as header parameters.
  • Secure the token path with the previous authentication mechanism.

Lo lets start.

Implementing a new Authentication Mechanism:

The symfony documentation explain to us how to implement an authentication mechanism using as example the WSSE specification, we will use the same tutorial for our classes so i will leave to you the documentation page and will show the final classes. Notice that i change the name of the classes from Wsse to Rest and make some tweaks like removing properties and parameters where are not needed.

AppBundle/Security/Authentication/Token/RestUserToken.php

<?php

namespace AppBundle\Security\Authentication\Token;

use Symfony\Component\Security\Core\Authentication\Token\AbstractToken;

class RestUserToken extends AbstractToken
{
    private $password;

    public function __construct($password)
    {
        $this->password = $password;
        parent::__construct();
    }

    public function getPassword()
    {
        return $this->password;
    }

    public function getCredentials()
    {
        return '';
    }
}

AppBundle/Security/Authentication/Provider/RestProvider.php

<?php

namespace AppBundle\Security\Authentication\Provider;

use AppBundle\Security\Authentication\Token\RestUserToken;
use Symfony\Component\Security\Core\Authentication\Provider\AuthenticationProviderInterface;
use Symfony\Component\Security\Core\Authentication\Token\TokenInterface;
use Symfony\Component\Security\Core\Exception\AuthenticationException;
use Symfony\Component\Security\Core\User\UserProviderInterface;

class RestProvider implements AuthenticationProviderInterface
{
    private $userProvider;

    public function __construct(UserProviderInterface $userProvider)
    {
        $this->userProvider = $userProvider;
    }

    public function authenticate(TokenInterface $token)
    {
        $user = $this->userProvider->loadUserByUsername($token->getUsername());

        if ($user && $user->getPassword() === $token->getPassword()) {
            $authenticatedToken = new RestUserToken($user->getRoles());
            $authenticatedToken->setUser($user);

            return $authenticatedToken;
        }

        throw new AuthenticationException('The REST authentication failed.');
    }

    public function supports(TokenInterface $token)
    {
        return $token instanceof RestUserToken;
    }
}

AppBundle/Security/Firewall/RestListener.php

<?php

namespace AppBundle\Security\Firewall;

use AppBundle\Security\Authentication\Token\RestUserToken;
use Symfony\Component\HttpKernel\Event\GetResponseEvent;
use Symfony\Component\Security\Core\Authentication\AuthenticationManagerInterface;
use Symfony\Component\Security\Core\Authentication\Token\Storage\TokenStorageInterface;
use Symfony\Component\Security\Core\Exception\AuthenticationException;
use Symfony\Component\Security\Http\Firewall\ListenerInterface;

class RestListener implements ListenerInterface
{
    protected $tokenStorage;
    protected $authenticationManager;

    public function __construct(TokenStorageInterface $tokenStorage, AuthenticationManagerInterface $authenticationManager)
    {
        $this->tokenStorage = $tokenStorage;
        $this->authenticationManager = $authenticationManager;
    }

    public function handle(GetResponseEvent $event)
    {
        $request = $event->getRequest();
        $username = $request->headers->get('username');
        $password = $request->headers->get('password');

        if (!$username && !$password) {
            return;
        }

        $token = new RestUserToken($password);
        $token->setUser($username);

        try {
            $authToken = $this->authenticationManager->authenticate($token);
            $this->tokenStorage->setToken($authToken);

            return;
        } catch (AuthenticationException $failed) {
            $token = $this->tokenStorage->getToken();
            if ($token instanceof RestUserToken) {
                $this->tokenStorage->setToken(null);
            }
            return;
        }
    }
}

AppBundle/DependencyInjection/Security/Factory/RestFactory.php

<?php

namespace AppBundle\DependencyInjection\Security\Factory;

use Symfony\Bundle\SecurityBundle\DependencyInjection\Security\Factory\SecurityFactoryInterface;
use Symfony\Component\Config\Definition\Builder\NodeDefinition;
use Symfony\Component\DependencyInjection\ContainerBuilder;
use Symfony\Component\DependencyInjection\DefinitionDecorator;
use Symfony\Component\DependencyInjection\Reference;

class RestFactory implements SecurityFactoryInterface
{
    public function create(ContainerBuilder $container, $id, $config, $userProvider, $defaultEntryPoint)
    {
        $providerId = 'security.authentication.provider.rest.' . $id;
        $container
            ->setDefinition($providerId, new DefinitionDecorator('rest.security.authentication.provider'))
            ->replaceArgument(0, new Reference($userProvider));

        $listenerId = 'security.authentication.listener.rest.' . $id;
        $listener = $container->setDefinition($listenerId, new DefinitionDecorator('rest.security.authentication.listener'));

        return array($providerId, $listenerId, $defaultEntryPoint);
    }

    public function getPosition()
    {
        return 'pre_auth';
    }

    public function getKey()
    {
        return 'rest';
    }

    public function addConfiguration(NodeDefinition $node)
    {
    }
}

Done, we have our classes in place, one thing to notice on the authentication method of the RestProvider class we validate that the password retrieved from the header is the same hashed password of the user that is trying to get his access token.

Lets configure the services on services.yml

rest.security.authentication.provider:
    class: AppBundle\Security\Authentication\Provider\RestProvider
    arguments: [""]
    public: false

rest.security.authentication.listener:
    class: AppBundle\Security\Firewall\RestListener
    arguments: ["@security.token_storage", "@security.authentication.manager"]
    public: false

Great all is in place just one extra configuration is required on the oauth_token firewall

oauth_token:
    pattern:      ^/oauth/v2/token
    rest:         true

Now we can try access get the access token as before and we will get an authentication error since we need to specify the username and hashed password on the header, example:

/oauth/v2/token?client_id=86_5sq6mu44srwo0kko8s4sckw40ksss8sw40wwgook40g0884ggw&client_secret=3er1dln3wa2o0oco0s44wkocwog0kg08sc80osck04wc4w44co&grant_type=password&redirect_uri=www.example.com&username=fabien&password=fabien

with header:

username: fabien
password: $2y$13$560vs8wzb2g40wg4cggs4uw66bA6G4VD4VW3KZnNmPNmGAe.JNRa2

And voila we got authenticated and receive the expected response

{
  access_token: "YzY2OWVlNDhjZDg0ZDhkMDEzNDg4ODhmOGU1MWM5YjZkYWNlYzEzODM5ZDc1N2JmZjJkZWUzNDZlMjExYmMwYQ"
  expires_in: 3600
  token_type: "bearer"
  scope: null
  refresh_token: "NTljMzhhMjRmOWY2NmFkNjAxMjM4YjlmMjdiOTIyNjQ1ODczZjI0NjZlZGVmOWZjOWI5NmQ0ZTQ4MmE4NDJkZA"
}

Conclusion:

Now we have a more secure RESTFul app that will allow us to build SPA and mobile Apps and expose our business rules so we can grow in the modern information era.

This method maybe is not what you really need, and exactly that is the point; as all the posts in this series the main point is to gave you some directions on what a RESTFul app should look so you can decide on your own if agree with it and build it.

I really hope this series has been helpful to you. Be free to write some comments or share it if you like it. Thanks for you reading time.

Series:

  1. Motivation
  2. REST Levels 0, 1, 2 ( FOSRestBundle, JMSSerializerBundle )
  3. REST Levels 0, 1, 2 ( FOSUserBundle )
  4. REST Levels 0, 1, 2 ( NelmioApiDocBundle )
  5. REST Levels 3 ( BazingaHateoasBundle )
  6. Security ( FOSOAuthServerBundle )
  7. Security ( Securing the token path )
  8. Security ( Securing the token path – FIXED )
Advertisements

Symfony a RESTFul app: Security ( FOSOAuthServerBundle )

In the previous post we set up a basic symfony project aiming to develop a RESTFul solution.

Now we are on the last post of this series, “Security“. You may question:

Why security as the last post, should not be security the first thing we should ensure?

Sure, security can be, and from my point of view; the most controversial and complex topic on our business. There a lot of ways to enforce security on our apps, ones may say some of those solutions are not bullet proof, others may say some of them are too complicated for their business. The truth is that you should study a lot about this topic and decide by your self which is the best solution for your business at the right moment.

Because all of that is why i leave this post for the last of this series so you can decide to read it or not, or even if it should be part of this series or not; i hope what ever your call is, this post remains useful.

As always all the code on this series can be found on this github repository under the symfony2_restful_example branch.

Securing a RESTFul App:

In almost all the cases you will need to perform some kind of user authentication and authorization, unless your app is completely open to anonymous, but for the sake of this post i will assume that you will need it.

From the many security approaches out there i will talk about only two of them, both i have used it on the past.

API Key Authentication and WSSE:

Both approaches share the same concept where the final authentication is performed with the API Token. In the first case is much basic since the clients need to know the token from the start and that can be used from 3th sources to attack our system. The WSSE specification describe a more secure process where the user authenticate with their credentials and then the API Token is delivered to the clients for farther use.

Both on this section are well documented on the symfony official documentation, so i will not extend on they and how to implement it.

OAuth:

As wikipedia describes:

OAuth is an open standard for authorization. OAuth provides client applications a ‘secure delegated access’ to server resources on behalf of a resource owner.

The standard evolve from version 1.0 to version 2.0

OAuth 2.0 focuses on client developer simplicity while providing specific authorization flows for web applications, desktop applications, mobile phones, and living room devices.

On this post we will talking about version 2.0 of OAuth since its more complete. This approach is used for authentication on almost all the social networks ans services that we use now a day (Google, Facebook, Twitter, LinkedIn, Github, etc)

FOSOAuthServerBundle:

Yes another bundle from FOS, this one will allow us to use OAuth2 on our projects. Sadly the documentation of this bundle is no so great as the others, and there are as always several ways to make it work; so i will focus on make it work for our little example, also i will try to explain the approach i choose to follow and the reasons why.

Lest start installing the bundle

composer require friendsofsymfony/oauth-server-bundle

And enable it on the AppKernel

new FOS\OAuthServerBundle\FOSOAuthServerBundle(),

Great we have the bundle installed. To work we need to define four models that in fact are the main terms of the OAuth2 specification, Client, AccessToken, RefreshToken and AuthCode. To keep following our clean architecture lets put this models inside our Customer package, they should look like this:

Client.php

<?php

namespace Customer;

use FOS\OAuthServerBundle\Entity\Client as BaseClient;

class Client extends BaseClient
{
    protected $id;
    protected $name;

    public function setName($name)
    {
        $this->name = $name;
    }

    public function getName()
    {
        return $this->name;
    }
}

I add to the client the property name so we can identify our client humanly, maybe on your business rules you can even find some other data like, the version of the client, the target platforms, etc.

AccessToken.php

<?php

namespace Customer;

use FOS\OAuthServerBundle\Entity\AccessToken as BaseAccessToken;

class AccessToken extends BaseAccessToken
{
    protected $id;
    protected $client;
    protected $customer;
}

RefreshToken.php

<?php

namespace Customer;

use FOS\OAuthServerBundle\Entity\RefreshToken as BaseRefreshToken;

class RefreshToken extends BaseRefreshToken
{
    protected $id;
    protected $client;
    protected $customer;
}

AuthCode.php

<?php

namespace Customer;

use FOS\OAuthServerBundle\Entity\AuthCode as BaseAuthCode;

class AuthCode extends BaseAuthCode
{
    protected $id;
    protected $client;
    protected $customer;
}

As you can see all the classes extend from the models exported by the bundle.

But we need to integrate it with symfony and doctrine so lets create our entities and the mapping.

AppBundle/Entity/Client.php

<?php

namespace AppBundle\Entity;

use Customer\Client as BaseClient;

class Client extends BaseClient
{
}

AppBundle/Entity/AccessToken.php

<?php

namespace AppBundle\Entity;

use Customer\AccessToken as BaseAccessToken;

class AccessToken extends BaseAccessToken
{
}

AppBundle/Entity/RefreshToken.php

<?php

namespace AppBundle\Entity;

use Customer\RefreshToken as BaseRefreshToken;

class RefreshToken extends BaseRefreshToken
{
}

AppBundle/Entity/AuthCode.php

<?php

namespace AppBundle\Entity;

use Customer\AuthCode as BaseAuthCode;

class AuthCode extends BaseAuthCode
{
}

AppBundle/Resources/config/doctrine/Client.orm.yml

AppBundle\Entity\Client:
    type: entity
    table: client
    id:
        id:
            id: true
            type: integer
            generator: { strategy: AUTO }
    fields:
        name:
            type: string
            lenght: 255
    lifecycleCallbacks: {  }

AppBundle/Resources/config/doctrine/AccessToken.orm.yml

AppBundle\Entity\AccessToken:
    type: entity
    table: access_token
    id:
        id:
            id: true
            type: integer
            generator: { strategy: AUTO }
    manyToOne:
        user:
            targetEntity: Customer
            joinColumn:
                name: costumer_id
                referencedColumnName: id
        client:
            targetEntity: Client
            joinColumn:
                name: client_id
                referencedColumnName: id
    lifecycleCallbacks: {  }

AppBundle/Resources/config/doctrine/RefreshToken.orm.yml

AppBundle\Entity\RefreshToken:
    type: entity
    table: refresh_token
    id:
        id:
            id: true
            type: integer
            generator: { strategy: AUTO }
    manyToOne:
        user:
            targetEntity: Customer
            joinColumn:
                name: costumer_id
                referencedColumnName: id
        client:
            targetEntity: Client
            joinColumn:
                name: client_id
                referencedColumnName: id
    lifecycleCallbacks: {  }

AppBundle/Resources/config/doctrine/AuthCode.orm.yml

AppBundle\Entity\AuthCode:
    type: entity
    table: auth_code
    id:
        id:
            id: true
            type: integer
            generator: { strategy: AUTO }
    manyToOne:
        user:
            targetEntity: Customer
            joinColumn:
                name: costumer_id
                referencedColumnName: id
        client:
            targetEntity: Client
            joinColumn:
                name: client_id
                referencedColumnName: id
    lifecycleCallbacks: {  }

Great we have our model set, lets finish the bundle configuration on config.yml.

fos_oauth_server:
    db_driver: orm
    client_class:        AppBundle\Entity\Client
    access_token_class:  AppBundle\Entity\AccessToken
    refresh_token_class: AppBundle\Entity\RefreshToken
    auth_code_class:     AppBundle\Entity\AuthCode
    service:
        user_provider: fos_user.user_manager

Note that is important to add the service for the user provider since we are using FOSUserBundle, if you have any other user provider you need to put it here.

I know a lot of steps but we are almost done, just two more configurations before we can actually start coding. Lets configure our security file security.yml, that is the main reason we are going through this post.

# To get started with security, check out the documentation:
# http://symfony.com/doc/current/book/security.html
security:
    encoders:
        FOS\UserBundle\Model\UserInterface: bcrypt

    providers:
        fos_userbundle:
            id: fos_user.user_provider.username

    firewalls:
        # disables authentication for assets and the profiler, adapt it according to your needs
        dev:
            pattern:      ^/(_(profiler|wdt)|css|images|js)/
            security:     false

        oauth_token:
            pattern:      ^/oauth/v2/token
            security:     false

        api_doc:
            pattern:      ^/api/doc
            security:     false

        api:
            pattern:      ^/api
            fos_oauth:    true
            stateless:    true

    access_control:
        - { path: ^/api, roles: [ IS_AUTHENTICATED_FULLY ] }

Lets talk about this configuration line by line:

encoders:
    FOS\UserBundle\Model\UserInterface: bcrypt

providers:
    fos_userbundle:
        id: fos_user.user_provider.username

We introduce the encoder and the user provider in the previous post about FOSUserBundle.

Firewalls, as we know; are the symfony way to secure routes by patterns so lets have a break down on the firewalls we have configured:

dev:
    pattern:      ^/(_(profiler|wdt)|css|images|js)/
    security:     false

The development default firewall so we can access the development profiler page and the static files, no security enabled.

api_doc:
    pattern:      ^/api/doc
    security:     false

api:
    pattern:      ^/api
    fos_oauth:    true
    stateless:    true

And the firewalls from the previous posts to access the api end points and the api documentation, notice that i add a prefix “api”, to the path; trying to differentiate it from the oauth ones. Also notice that we set the authentication on the api route to be ensure with the FOSOAuthServerBundle.

oauth_token:
    pattern:      ^/oauth/v2/token
    security:     false

I leave to the final the oauth token end point, since this can be the controversial point. As we can see this end point do not have any security enabled, this is the default configuration provided by the bundle documentation. What this end point does is retrieve the data from the request and validate the client with the public id and the secret, as well the redirect uri, the grant type, the scope and lastly the user with the username and password.

This last part its what makes me doubt of the security since the username and the password are been sent as plain text on the request, and from my point of view this is not a good idea.

For the sake of the post lets leave this discussion for the final part of the post series, Secure the token path; and lets keep going with the configuration and coding.

access_control:
    - { path: ^/api, roles: [ IS_AUTHENTICATED_FULLY ] }

Finally in the access control section lets secure our api to be only accessed by fully authenticated users.

Great we have our security mechanism in place, lets test it. But first we need a client and our user authorize that client. The bundle comes with an implementation for that authorization part, that can be configured to be anything you need, its basically a page like the ones from facebook and google, where the user allow the application to gave authentication for it.

So, Why we don’t include it on our example?

Since this post series its about a RESTFul app, there is no view where the user see this allowing page, going even farther our internal API should not be allowed from our user, lets say we are not exporting our API to third party developers, so our app clients should be always allowed for our clients. For that matter i create a command that allow us to create new clients and authorized them for all our users, lets see it.

<?php
namespace AppBundle\Command;

use Symfony\Bundle\FrameworkBundle\Command\ContainerAwareCommand;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\HttpFoundation\Request;

class CreateOAuthClientCommand extends ContainerAwareCommand
{
    protected function configure()
    {
        $this
            ->setName('oauth:client:create')
            ->setDescription('Create OAuth Client')
            ->addArgument(
                'name',
                InputArgument::REQUIRED,
                'Client Name?'
            )
            ->addArgument(
                'redirectUri',
                InputArgument::REQUIRED,
                'Redirect URI?'
            )
            ->addArgument(
                'grantType',
                InputArgument::REQUIRED,
                'Grant Type?'
            );
    }

    protected function execute(InputInterface $input, OutputInterface $output)
    {
        $container = $this->getContainer();
        $oauthServer = $container->get('fos_oauth_server.server');

        $name = $input->getArgument('name');
        $redirectUri = $input->getArgument('redirectUri');
        $grantType = $input->getArgument('grantType');

        $clientManager = $container->get('fos_oauth_server.client_manager.default');
        $client = $clientManager->createClient();
        $client->setName($name);
        $client->setRedirectUris([$redirectUri]);
        $client->setAllowedGrantTypes([$grantType]);
        $clientManager->updateClient($client);

        $output->writeln(sprintf("<info>The client <comment>%s</comment> was created with <comment>%s</comment> as public id and <comment>%s</comment> as secret</info>",
            $client->getName(),
            $client->getPublicId(),
            $client->getSecret()));

        $customers = $container->get('doctrine')->getRepository('AppBundle:Customer')->findAll();

        foreach ($customers as $customer) {
            $queryData = [];
            $queryData['client_id'] = $client->getPublicId();
            $queryData['redirect_uri'] = $client->getRedirectUris()[0];
            $queryData['response_type'] = 'code';
            $authRequest = new Request($queryData);

            $oauthServer->finishClientAuthorization(true, $customer, $authRequest, $grantType);

            $output->writeln(sprintf("<info>Customer <comment>%s</comment> linked to client <comment>%s</comment></info>",
                $customer->getId(),
                $client->getName()));
        }
    }
}

What do we did here, this command receive the client name, the redirect uri and the grant type, after create the client it retrieve all the users/customers and authorize the recently created app to all. After the execution of this command we should be able to request an access token for a user and then authenticate to query the API. Lest see it on an example:

app/console oauth:client:create client1 www.client1.com code

We create an application named client1, should see an output like this:

The client client1 was created with 88_5qhmvel4ozoko8ws0gc8404s40s40k040k04s8okoo44wcww40 as public id and 4431nwlagk8wsok00sogkkwscgk44k8g484ow04c8o4cwc000s as secret
Customer 55f0369e0cdac linked to client client1

Now we can navigate to the token end point and request the access token

/oauth/v2/token?client_id=89_6488j8o9pickwwgwgos800gkws8o0wk8kwcs84skoo4c04gksw&client_secret=5vmzu0sro10kw8sosg4ccoco4ccs48wwog0kkcgwoksw800s0&grant_type=password&redirect_uri=www.client1.com&username=fabien&password=fabien

As you can see we need to include the username and password of the user in plain text. The response of this request should look like this:

{"access_token":"NDdhYTRjMmQ3MmNmZjZjOWFjYTA5NDQwZmZhNTgwMjczYzYyMDg5ODYzMWZmZjQ2MGUxMDJmNjNmNzI5Zjk4ZA","expires_in":3600,"token_type":"bearer","scope":null,"refresh_token":"YWIzMThhZDY1MGQzZGRlYjlhOGRiYTJjZTdmZjZmMWUyY2Y0YmUyZjRhMTU0YjE5N2VjYzcwZDQ4OTZlZDdlYg"}

Finally we can use the access token to query the api, lets try our old customers end point:

/api/customers

With the header:

Authorization: Bearer NDdhYTRjMmQ3MmNmZjZjOWFjYTA5NDQwZmZhNTgwMjczYzYyMDg5ODYzMWZmZjQ2MGUxMDJmNjNmNzI5Zjk4ZA

And we should see the expected response:

{
id: "55f0369e0cdac"
username: "fabien"
username_canonical: "fabien"
email: "fabien@symfony.com"
email_canonical: "fabien@symfony.com"
enabled: true
salt: "aru3jkvy1r4gcoock4gwocko0w8w4sk"
password: "$2y$13$aru3jkvy1r4gcoock4gwoO2/lUF5PVlUHOqJIUCX9geX7jZxxjTfC"
locked: false
expired: false
roles:
  [0]
credentials_expired: false
links: {
  self: {
    href: "/customers/55f0369e0cdac"
    }
  customers: {  
    href: "/customers"  
    } 
  } 
}

Till now we have successfully configured an OAuth2 service, great.

Conclusion:

Security is really a hard topic to talk about, but i hope this post point you to some ideas on how to accomplish it. OAuth2 is a great protocol, i always think that if the big companies are using something, that something suppose to be good ;).

At the end i decide to add one more post to the series to talk about and of course show some code on what solution do i apply to solve the security flaw on the token path.

Series:

  1. Motivation
  2. REST Levels 0, 1, 2 ( FOSRestBundle, JMSSerializerBundle )
  3. REST Levels 0, 1, 2 ( FOSUserBundle )
  4. REST Levels 0, 1, 2 ( NelmioApiDocBundle )
  5. REST Levels 3 ( BazingaHateoasBundle )
  6. Security ( FOSOAuthServerBundle )
  7. Security ( Securing the token path )
  8. Security ( Securing the token path – FIXED )

Symfony a RESTFul app: REST Levels 3 ( BazingaHateoasBundle )

In the previous post we set up a basic symfony project aiming to develop a RESTFul solution.

Great we are now on the last level of the REST model, the Hypermedia Controls. What are Hypermedia Controls, as Martin Fowler explains:

There’s no absolute standard as to how to represent hypermedia controls.

So this post its about how we can deliver this Hypermedia Controls on our RESTFul apps using the proposal from Martin Fowler and Leonard Richardson.

As always all the code on this series can be found on this github repository under the symfony2_restful_example branch.

HATEOAS:

In the article Martin Fowler say:

HATEOAS (Hypertext As The Engine Of Application State). It addresses the question of how to get from a list open slots to knowing what to do to book an appointment.

With this links the client can follow the directions on what to do with the given resource even if they don’t know the documentation of your API.

Since the Hypermedia Control links should be relative they allow the flexibility to change the domain holding the api without braking the map.

There is an implementation of this specification for PHP you can find it at http://hateoas-php.org/ and of course the symfony bundle:

BazingaHateoasBundle:

This bundle is developed by William Durand the same developer of the base hateoas php library. It allow us to integrate this Hypermedia Control in our REST APIs. The bundle is well documented you can check it on https://github.com/willdurand/BazingaHateoasBundle/blob/master/Resources/doc/index.md

Lets install it using composer

composer require willdurand/hateoas-bundle

And enable it on the AppKernel

new Bazinga\Bundle\HateoasBundle\BazingaHateoasBundle(),

As described on the doc this bundle require the JMSSerializerBundle but since we already have it installed from the previous posts there is nothing extra to do.

At this point we have the bundle configured so lets map some Hypermedia Controls on our entities, lets use the Customer

<?php

namespace AppBundle\Entity;

use Customer\Customer as CustomerBase;
use Hateoas\Configuration\Annotation as Hateoas;
use JMS\Serializer\Annotation as Serializer;

/**
 * Customer ORM Entity
 *
 * @Serializer\XmlRoot("customer")
 *
 * @Hateoas\Relation("self", href = @Hateoas\Route("get_customers", parameters = { "customer" = "expr(object.getId())" }))
 * @Hateoas\Relation("customers", href = @Hateoas\Route("cget_customers"))))
 */
class Customer extends CustomerBase
{
}

We are defining two relations, one with it self and one with the collection so the client can easily navigate through our api following the directions we gave.

One call to the end point for an specific customer, lets take this example /customers/55eeec8f80c0f should respond with something like this:

{
id: "55eeec8f80c0f"
username: "fabien"
username_canonical: "fabien"
email: "fabien@symfony.com"
email_canonical: "fabien@symfony.com"
enabled: true
salt: "aru3jkvy1r4gcoock4gwocko0w8w4sk"
password: "$2y$13$aru3jkvy1r4gcoock4gwoO2/lUF5PVlUHOqJIUCX9geX7jZxxjTfC"
locked: false
expired: false
roles:
  [0]
credentials_expired: false
links: {
  self: {
    href: "/customers/55eeec8f80c0f"
    }
  customers: {
    href: "/customers"
    }
  }
}

The library expose several annotations and helpers that we can use to build this expressive Hypermedia Control map, but for the sake of the extensiveness of this post i will leave it to you.

Conclusion:

Following the good practices and applying for all the levels of the REST model, we manage to build a simple RESTFul app, sure its really simple at the point of useless, but the aim of this post series is to demonstrate how a RESTFul app should look alike; again your business rules will guide you through the process on what to build but sure this practices will lead you to a great RESTFul app.

One more post remains to talk about security, this is topic where there is not a single line write on stone, there are so many ways to implement it, that post will be just a single reference on how to put it all together and an advice to use OAuth 2.0.

Series:

  1. Motivation
  2. REST Levels 0, 1, 2 ( FOSRestBundle, JMSSerializerBundle )
  3. REST Levels 0, 1, 2 ( FOSUserBundle )
  4. REST Levels 0, 1, 2 ( NelmioApiDocBundle )
  5. REST Levels 3 ( BazingaHateoasBundle )
  6. Security ( FOSOAuthServerBundle )
  7. Security ( Securing the token path )
  8. Security ( Securing the token path – FIXED )

Symfony a RESTFul app: REST Levels 0, 1, 2 ( NelmioApiDocBundle )

In the previous post we set up a basic symfony project aiming to develop a RESTFul solution.

Its nice to develop great applications with the tools we love, been well supported and with an strong community behind it, its even more satisfactory to develop this apps real fast and with good software architecture in mind so future contributors can get to theirs work with less effort. But what happens with our clients?

In this case the clients of RESTFul apps are also developers that need to integrate web based SPA or mobile applications. We need to take care of those clients too and documentation its a really important in those cases. Not just documentation, but a well described and updated documentation.

As always all the code on this series can be found on this github repository under the symfony2_restful_example branch.

NelmioApiDocBundle:

This is another great bundle delivered from the community to the community. It expose configurations and enable features that allow us to deliver REST API Documentation in a really simple and nicest way. The documentation for this bundle can be found on this page.

After installation and configuration, following the instructions on the bundle documentation; we can navigate to the base url of our API and add /api to it, then we should see our first auto-generated documentation page.

Lest add some info to that page, lets document the Greeting Controller

/**
 * Response with a nice "Hello" greeting message
 *
 * @ApiDoc(
 *  section="Greetings",
 *  resource=true,
 *  description="Response with a Hello greeting",
 *  statusCodes={
 *         200="Returned when successful"
 *  },
 *  tags={
 *   "stable" = "#4A7023",
 *  }
 * )
 */

You can see how easy its to document an action, but also grouping it by resources and sections, even farther tagging the end point so the clients can filter for it or be careful in some cases, rise deprecated notices and so on.

Lets see another example on the Customer Controller

/**
 * Response with the customer that has {customer} for id
 *
 * @ApiDoc(
 *  section="Customer",
 *  description="Get a customer",
 *  requirements={
 *      {
 *          "name"="customer",
 *          "dataType"="string",
 *          "requirement"="*",
 *          "description"="customer id"
 *      }
 *  },
 *  statusCodes={
 *         200="Returned when successful"
 *  },
 *  tags={
 *   "stable" = "#4A7023",
 *   "need validations" = "#ff0000"
 *  }
 * )
 */

This one will be the documentation for the get customer action. Even if the bundle its smart enough to collect the action parameters and include it on the documentation, example the {customer} parameter that suppose to be the customer id, the auto-generated page will not include any farther info about it, and remember that documentation should be as expressive as possible, that its why i manually include the specification for the {customer} parameter.

Lastly what about actions that receive data such as posts and puts

/**
 * Create a new customer
 *
 * @ApiDoc(
 *  section="Customer",
 *  description="Create a new Customer",
 *  input="AppBundle\Form\CustomerType",
 *  output="AppBundle\Entity\Customer",
 *  statusCodes={
 *         200="Returned when successful"
 *  },
 *  tags={
 *   "stable" = "#4A7023",
 *   "need validations" = "#ff0000"
 *  },
 *  views = { "premium" }
 * )
 */

This one is the annotation for the new customer action, you can see the input parameter with the form that we will use to capture the data and the output with the suppose response definition of the expected object, that’s great. There is one more option in this annotation and is the views, suppose that only premium clients are allow to create new customers, and lets forget for a moment that we need to validate and secure the action; the bundle allow you through the views option to configure multiple views for each client so you can expose to each client what you need, even if its another grouping option its more powerful.

As explained before this bundle will extract the documentation from the same code we use to deliver the functionalities, inferring as much as possible from the classes names, function names, parameter names and so on.

Conclusion:

Documentation its really important for those clients that suppose to use your API, so thanks to this great bundle we can manage in an easy and fast way deliver it.

With this we manage to cover the 3 first levels of a RESTFul app, we are getting close to the target.

Series:

  1. Motivation
  2. REST Levels 0, 1, 2 ( FOSRestBundle, JMSSerializerBundle )
  3. REST Levels 0, 1, 2 ( FOSUserBundle )
  4. REST Levels 0, 1, 2 ( NelmioApiDocBundle )
  5. REST Levels 3 ( BazingaHateoasBundle )
  6. Security ( FOSOAuthServerBundle )
  7. Security ( Securing the token path )
  8. Security ( Securing the token path – FIXED )

Symfony a RESTFul app: REST Levels 0, 1, 2 ( FOSUserBundle )

In the previous post we set up a basic symfony project aiming to develop a RESTFul solution.

Real projects need to work with databases and persistent data so this post will add this new interaction with the basic entities of all the users.

As always all the code on this series can be found on this github repository under the symfony2_restful_example branch.

Customer:

Lets define our Customer model following the clean architecture on a directory called Customer. Why Customer and not User, since the terminology of user can be confuse some times, and because “user” is a reserved word on common databases; it will be better to use a more clear term, you can change it to something else if you feel it will be better, or feet your business bests.

<?php

namespace Customer;

use FOS\UserBundle\Model\User as BaseUser;

/**
 * Customer
 */
class Customer extends BaseUser
{

    /**
     * @var string
     */
    protected $id;

    /**
     * Customer constructor.
     * @param $id
     */
    public function __construct()
    {
        $this->id = $this->id ? $this->id : uniqid();
        parent::__construct();
    }

    /**
     * Get id
     *
     * @return string
     */
    public function getId()
    {
        return $this->id;
    }
}

and add it to the autoload

$loader->addPsr4("Customer\\", __DIR__.'/../../Customer', true);

Now lets integrate that model with doctrine creating an entity class on the Entity directory under the bundle

use Customer\Customer as CustomerBase;

/**
 * Customer ORM Entity
 */
class Customer extends CustomerBase
{
}

and the mapping on the Resources/config/doctrine under the bundle

AppBundle\Entity\Customer:
    type: entity
    table: null
    repositoryClass: AppBundle\Entity\CustomerRepository
    id:
        id:
            type: string
            length: 255
    lifecycleCallbacks: {  }

This is a simple customer model with only the id. You may notice that the id its an string, and not the usual integer auto-incremental; this its because we will be using UUID, this is a good practice i learn from Mathias Verraes, for that notice that the constructor will set that id.

FOSUserBundle:

This great bundle comes to be so popular that even have its own page on the symfony documentation. It expose configurations and enable features that allow us to build the common interaction of users, with our systems; on symfony in a quick and easy way. So lets follow the steps described on the page.

First lets enable the bundle on the AppKernel, then lets configure the bundle with the basic configuration:

fos_user:
    db_driver: orm
    firewall_name: main
    user_class: AppBundle\Entity\Customer

Finally we need to all the fields that suppose to be on a customer/user table, FOSUserBundle propose to use to use and extends their common base class, its a complete structure that define a common users interaction and implements the Symfony user Interfaces. But this class has one back draw, it requires and use Collection and ArrayCollection from Doctrine, this its not a good implementation of the DI principle, since we will be now depending our whole architecture from Doctrine’s. The class should look like this:

<?php

namespace Customer;

use FOS\UserBundle\Model\User as BaseUser;

/**
 * Customer
 */
class Customer extends BaseUser
{

    /**
     * @var string
     */
    protected $id;

    /**
     * Get id
     *
     * @return string
     */
    public function getId()
    {
        return $this->id;
    }
}

After this we can execute the doctrine command to create our schema with:

doctrine:schema:create

Even if we are not going to use any authentication method yet we need to configure some security sections so FOSUserBundle can create our customers/users, our security.yml end like this:

# To get started with security, check out the documentation:
# http://symfony.com/doc/current/book/security.html
security:
    encoders:
        FOS\UserBundle\Model\UserInterface: bcrypt

    providers:
        fos_userbundle:
            id: fos_user.user_provider.username

    role_hierarchy:
        ROLE_ADMIN:       ROLE_USER
        ROLE_SUPER_ADMIN: ROLE_ADMIN

    firewalls:
        # disables authentication for assets and the profiler, adapt it according to your needs
        dev:
            pattern: ^/(_(profiler|wdt)|css|images|js)/
            security: false

        main:
            anonymous: ~
            # activate different ways to authenticate

            # http_basic: ~
            # http://symfony.com/doc/current/book/security.html#a-configuring-how-your-users-will-authenticate

            # form_login: ~
            # http://symfony.com/doc/current/cookbook/security/form_login_setup.html

Now lets create our first costumer with the FOSUserBundle command

fos:user:create

This command will prompt us for the user username, email and password, use the data you like.

Ok now we have the bundle correctly configured and our first customer on db, lets start building our REST API. To start lets create a basic end point to get all the registered customers.

Create a CustomersController and inside it create a getAction that will query the db to find all the registered customers

<?php

namespace AppBundle\Controller;

use FOS\RestBundle\Controller\Annotations as FOSRestBundleAnnotations;
use FOS\RestBundle\Controller\FOSRestController;
use FOS\RestBundle\Routing\ClassResourceInterface;

/**
 * @FOSRestBundleAnnotations\View()
 */
class CustomersController extends FOSRestController implements ClassResourceInterface
{
    public function cgetAction()
    {
        $em = $this->getDoctrine()->getEntityManager();
        $repository = $em->getRepository("AppBundle:Customer");
        $customers = $repository->findAll();
        return $customers;
    }
}

Sure we can filter that query to get only the enabled ones but this suppose to be a simple example, your business requirement will lead you to what ever you really needs.

After query the end point at “/customers” you should see something like this:

[
 {
 "id":"55edc9c58b131",
 "username":"fabien",
 "username_canonical":"fabien",
 "email":"fabien@symfony.com",
 "email_canonical":"fabien@symfony.com",
 "enabled":true,
 "salt":"tr24is47dj4wgkcoc0kockgg0c80wgk",
 "password":"$2y$13$tr24is47dj4wgkcoc0kocea5.mMJOVBgQAqND.no.ad7bSEWzWTxi",
 "locked":false,
 "expired":false,
 "roles":[],
 "credentials_expired":false
 }
]

Add a new action to retrieve a customer by id its as simple as using the symfony param converter

public function getAction(Customer $customer)
{
    return $customer;
}

Lets try a hard one and create a new customer through the API. What we need its a form that handles the customer basic data and a new end point that can process that form.

<?php

namespace AppBundle\Form;

use Symfony\Component\Form\AbstractType;
use Symfony\Component\Form\FormBuilderInterface;
use Symfony\Component\OptionsResolver\OptionsResolver;

class CustomerType extends AbstractType
{
    /**
     * @param FormBuilderInterface $builder
     * @param array $options
     */
    public function buildForm(FormBuilderInterface $builder, array $options)
    {
        $builder
            ->add('username', 'text')
            ->add('email', 'email')
            ->add('password', 'password');
    }

    /**
     * @param OptionsResolver $resolver
     */
    public function configureOptions(OptionsResolver $resolver)
    {
        $resolver->setDefaults(array(
            'data_class' => 'AppBundle\Entity\Customer',
            'csrf_protection' => false
        ));
    }

    /**
     * @return string
     */
    public function getName()
    {
        return '';
    }
}
public function postAction(Request $request)
{
    $customer = new Customer();
    $customerForm = $this->createForm(new CustomerType(), $customer);

    $customerForm->handleRequest($request);

    if ($customerForm->isValid()) {
        $em = $this->getDoctrine()->getEntityManager();
        $em->persist($customer);
        $em->flush();
        return $customer;
    }

    return $customerForm->getErrors();
}

A simple form and controller action, if you try post to the end point /customers with:

{
 "username": "yasmany",
 "email": "yasmanycm@gmail.com",
 "password": "ok"
}

You should get a response like

{
  id: "55edde808316d"
  username: "yasmany"
  username_canonical: "yasmany"
  email: "yasmanycm@gmail.com"
  email_canonical: "yasmanycm@gmail.com"
  enabled: false
  salt: "q8k22c6q2c0cs008s04wgsggwsgco0w"
  password: "ok"
  locked: false
  expired: false
  roles: [0]
  credentials_expired: false
}

Great we can now add new customers to our database. You may notice that the code does not perform any error validation, again this suppose to be a simple example and your business rules will guide you on that process.

Conclusion:

Our REST API its growing, so far we manage to create some basics end points and have it working easily enough with the help of this awesome bundles. Lets keep in mind the big picture:

  • Have a REST API that follow the 4 levels of a RESTFul app
  • To have a clean architecture where we can separate the framework from the business

So far our end points define resources and we use the HTTP verbs to interact with our resources, we may say, till now; “mission accomplish”, so lets keep moving.

Series:

  1. Motivation
  2. REST Levels 0, 1, 2 ( FOSRestBundle, JMSSerializerBundle )
  3. REST Levels 0, 1, 2 ( FOSUserBundle )
  4. REST Levels 0, 1, 2 ( NelmioApiDocBundle )
  5. REST Levels 3 ( BazingaHateoasBundle )
  6. Security ( FOSOAuthServerBundle )
  7. Security ( Securing the token path )
  8. Security ( Securing the token path – FIXED )

Symfony a RESTFul app: REST Levels 0, 1, 2 ( FOSRestBundle, JMSSerializerBundle )

REST Levels 0,1,2:

As Martin Fowler explains about Leonard Richardson REST model the first three levels are bout to use the resources provided by the HTTP Protocol to communicate actions in a more verbose way. Using URLs as Resources to uniquely identify it by identity. Using HTTP Request Method as action verb, so a request to a Resource /posts can be one of this five:

GET http://domain/posts                 // get a collection of posts
GET http://domain/posts/1            // get the post id 1
POST http://domain/posts             // create a new post
PUT http://domain/posts/1           // update the post id 1
PATCH http://domain/posts/1     // update the post id 1
DELETE http://domain/posts/1    // delete the post id 1

There is a difference between the PUT and PATCH methods, as explained on this article; The PUT action should replace the whole Resource with the new information provided, instead the PATCH action should replace only the fields with the provided information.

In this post we will try to create an example using symfony2. As always all the code on this series can be found on this github repository under the symfony2_restful_example branch.

FOSRestBundle:

This great bundle comes to be so popular that even have its own page on the symfony documentation. It expose configurations and enable features that allow us to build REST applications with symfony in a quick and easy way. So lets follow the steps described on the page.

Installation, follow the easy steps on the documentation page. For this bundle to work a serializer is required and the proposed one its JMSSerializerBundle.

Until now only installation and including the bundles on the AppKernel.php has been made.

Configuration:

fos_rest:
    disable_csrf_role: ROLE_API
    param_fetcher_listener: true
    body_listener: true
    allowed_methods_listener: true
    unauthorized_challenge: "Basic realm=\"Restricted Area\""
    access_denied_listener:
        json: true
        xml: true
        html: true
    view:
        view_response_listener: force
        force_redirects:
          html: true
        formats:
            json: true
            xml: true
    format_listener:
        rules:
            - { path: ^/, priorities: [ json, xml ], fallback_format: json, prefer_extension: true }

What this configuration means, a line by line break down:

disable_csrf_role: ROLE_API

Disable the csrf validation on form submissions only if the authenticated user have the ROLE_API assigned.

param_fetcher_listener: true

Enable the symfony param fetcher strategy

body_listener: true

Decode the data sent in the request so the Request Object can be populated with the desired method like application/x-www-form-urlencode or application/json.

allowed_methods_listener: true

adds the Allow HTTP header to each request appending all allowed methods for a given resource.

unauthorized_challenge: "Basic realm=\"Restricted Area\""
access_denied_listener:
    json: true
    xml: true
    html: true

This listener will enter before the symfony firewall exception listener and handle the response for all the configured formats.

view:
    view_response_listener: force
    force_redirects:
      html: true
    formats:
        json: true
        xml: true

The view listener its the most complex so i will leave it to the documentation. As a quick explanation it handle the response from the controller as a View object and finally converted as the expected format.

format_listener:
    rules:
        - { path: ^/, priorities: [ json, xml ], fallback_format: json, prefer_extension: true }

Finally the format listener set the rules to be taken into consideration on the routes that match the path regex, on what response formats are allowed, the priority to choose them if detection fails, if it will have a fallback format by default and much more.

This is a really simple configuration, the bundle expose a more complete one that allow you to configure your project at the specific level you need, but the explanation of the configuration its not on the scope of this post.

Routing:

greetings:
    resource: "AppBundle\Controller\GreetingsController"
    type:     rest

This way we are telling symfony to use the FOSRestBundle Routing generator that will parse our controller class and generate routes for all our public functions with the HTTP Method that is defined on the method name, more about this can be found on the documentation.

Lets add a simple model called Hello following the clean architecture on a directory called Greetings

/**
 * Class Hello
 */
class Hello
{
    /**
     * @var string
     */
    private $greet;

    /**
     * Hello constructor.
     */
    public function __construct()
    {
        $this->greet = "Hello World!!!";
    }

    /**
     * @return string
     */
    public function __toString()
    {
        return $this->greet;
    }
}

And add it to the autoload

$loader->addPsr4("Greetings\\", __DIR__.'/../../Greetings', true);

Our controller looks like this, using or Hello model

<?php

namespace AppBundle\Controller;

use FOS\RestBundle\Controller\Annotations as FOSRestBundleAnnotations;
use FOS\RestBundle\Controller\FOSRestController;
use FOS\RestBundle\Routing\ClassResourceInterface;
use Greetings\Hello;

/**
 * @FOSRestBundleAnnotations\View()
 */
class GreetingsController extends FOSRestController implements ClassResourceInterface
{
    public function helloAction()
    {
        return new Hello();
    }
}

This will generate the following path:

hello_greetings          GET    ANY    ANY  /greetings/hello.{_format}

If we request that url we should see the following response:

{
  greet: "Hello World!!!"
}

What gave us this configuration:

  • A set of rules for the routes that will follow the level 1 of the REST model
  • Easy way to handle several response formats like json and xml

Conclusion:

At this point our solution follow the first three levels of the REST model, FOSRestBundle enable us to easily start building a REST app with good practices.

Series:

  1. Motivation
  2. REST Levels 0, 1, 2 ( FOSRestBundle, JMSSerializerBundle )
  3. REST Levels 0, 1, 2 ( FOSUserBundle )
  4. REST Levels 0, 1, 2 ( NelmioApiDocBundle )
  5. REST Levels 3 ( BazingaHateoasBundle )
  6. Security ( FOSOAuthServerBundle )
  7. Security ( Securing the token path )
  8. Security ( Securing the token path – FIXED )

Symfony a RESTFul app: Motivation

This post series its about how to build a RESTFul app with Symfony2. There is a whole project about it, but i think its kind of outdated; but this series is about how do it by your self layer by layer. RESTFul apps are more and more popular this days. In my opinion a better approach to build your app, this enable you to build SPA and prepare your self to the mobile world, today business that do not target mobile apps are limiting the scope and the expansion of the app.

RESTFul the concept:

As defined on wikipedia:

The architectural properties of REST are realized by applying specific interaction constraints to components, connectors, and data elements. One can characterise applications conforming to the REST constraints described in this section as “RESTful”. If a service violates any of the required constraints, it cannot be considered RESTful. Complying with these constraints, and thus conforming to the REST architectural style, enables any kind of distributed hypermedia system to have desirable non-functional properties, such as performance, scalability, simplicity, modifiability, visibility, portability, and reliability.

The glory of REST:

overview

In an article by  about the model developed by Leonard Richardson it describe the break down of the levels of a RESTFul app. This post series will also aim to reach the glory of rest.

Code Requirements:

NOTE: This series post will require you to know basic symfony and composer commands usage as well as symfony terminologies.
NOTE: Each post on the series will own its own branch, for the sake of this post i will use BRANCH_NAME to refer the branch you should checkout on each post

Clone the project

git clone git@github.com:bitgandtter/k8s_php_test.git
git branch BRANCH_NAME
cd k8s_php_test

Build image

bash development_tools/cluster_config/app/build-image.sh

Create kubernetes replication controller and services for app and mysql

bash development_tools/cluster_config/app/create.sh
bash development_tools/cluster_config/mysql/create.sh

Update symfony vendors

cd framework
composer update

Ensure cache and logs permissions

chmod -R 777 app/cache
chmod -R 777 app/logs

Check kubernetes services ips

bash development_tools/cluster_config/get_pods_and_services.sh

With this steps you should be set to start hacking.

This series will be composed of this articles that im going to be writing in the upcoming days:

  1. Motivation
  2. REST Levels 0, 1, 2 ( FOSRestBundle, JMSSerializerBundle )
  3. REST Levels 0, 1, 2 ( FOSUserBundle )
  4. REST Levels 0, 1, 2 ( NelmioApiDocBundle )
  5. REST Levels 3 ( BazingaHateoasBundle )
  6. Security ( FOSOAuthServerBundle )
  7. Security ( Securing the token path )
  8. Security ( Securing the token path – FIXED )