=================== Resource Management =================== .. sectionauthor:: Christian Müller Traditionally a PHP application deals directly with all kinds of files. Realizing a file upload is usually an excessive task because you need to create a proper upload form, deal with deciphering the ``$_FILES`` superglobal and move the uploaded file from the temporary location to a safer place. You also need to analyze the content (is it safe?), control web access and ultimately delete the file when it's not needed anymore. Flow relieves you of this hassle and lets you deal with simple ``PersistentResource`` instances instead. File uploads are handled automatically, enforcing the restrictions which were configured by means of validation rules. The publishing mechanism was designed to support a wide range of scenarios, starting from simple publication to the local file system up to fine grained access control and distribution to one or more content delivery networks. This all works without any further ado by you, the application developer. Storage ======= The file contents belonging to a specific ``PersistentResource`` need to be stored in some place, they are not stored in the database together with the object. Applications should be able to store this content in several places as needed, therefor the concept of a *Storage* exists. A *Storage* is configured via ``Settings.yaml``: .. code-block:: yaml Neos: Flow: resource: storages: defaultPersistentResourcesStorage: storage: 'Neos\Flow\ResourceManagement\Storage\WritableFileSystemStorage' storageOptions: path: '%FLOW_PATH_DATA%Persistent/Resources/' The configuration for the ``defaultPersistentResourceStorage`` (naming for further storages is up to the developer) uses a specific Storage implementation class that abstracts the operations needed for a storage. In this case it is the ``WritableFileSystemStorage`` which stores data in a given ``path`` on the local file system of the application. Custom implementations allow you to store their resource contents in other places as needed. You can configure as many storages as you want to separate different types of resources, like your users avatars, generated invoices or any other type of resource you have. Flow comes configured with two storages by default: * *defaultStaticResourcesStorage* is the storage for static resources from your packages. This storage is readonly and does not operate on ``PersistentResource`` instances. See additional information about package resources below. * *defaultPersistentResourcesStorage* is the general storage for ``PersistentResource`` content. This storage is used as default if nothing else is specified. Custom storages will most likely be similar to this storage so all of the information below applies. Target ====== Flow is a web application framework and as such some (or most) of the resources in the system need to be made accessible online. The resource storages are not meant to be accessible so a ``Target`` is a configured way of telling how resources are to be published to the web. The default target for our persistent storage above is configured like this: .. code-block:: yaml Neos: Flow: resource: targets: localWebDirectoryPersistentResourcesTarget: target: 'Neos\Flow\ResourceManagement\Target\FileSystemSymlinkTarget' targetOptions: path: '%FLOW_PATH_WEB%_Resources/Persistent/' baseUri: '_Resources/Persistent/' This configures the ``Target`` named ``localWebDirectoryPersistentResourcesTarget``. Resources using this target will be published into the the given ``path`` which is inside the public web folder of Flow. The class ``Neos\Flow\ResourceManagement\Target\FileSystemSymlinkTarget`` is the implementation responsible for publishing the resources and providing public URIs to it. From the name you can guess that it creates symlinks to the resources stored on the local filesystem to save space. Other ``Target`` implementations could publish the resources to CDNs or other external locations that are publicly accessible. If you have lots of resources in your project you might run into problems when executing ``./flow resource:publish`` since the number of folders can be limited depending on the file system you're using. An error that might occur in this case is "Could not create directory". To circumvent this error you can tell Flow to split the resources into multiple subfolders in the ``_Resources/Persistent`` folder of your Web root. The option for your Target you need to set in this case is ``subdivideHashPathSegment: true``. .. code-block:: yaml Neos: Flow: resource: targets: localWebDirectoryPersistentResourcesTarget: target: 'Neos\Flow\ResourceManagement\Target\FileSystemSymlinkTarget' targetOptions: path: '%FLOW_PATH_WEB%_Resources/Persistent/' baseUri: '_Resources/Persistent/' subdivideHashPathSegment: true Collections =========== Flow bundles your ``PersistentResource``s into collections to allow separation of different types of resources. A ``Collection`` is the binding between a ``Storage`` and a ``Target`` and each ``PersistentResource`` belongs to exactly one ``Collection`` and by that is stored in the matching storage and published to the matching target. You can configure as many collections as you need for specific parts of your application. Flow comes preconfigured with two default collections: * *static* which is the collection using the ``defaultStaticResourcesStorage`` and ``localWebDirectoryStaticResourcesTarget`` to work with (static) package resources. This Collection is meant read-only, which is reflected by the storage used. In this Collection all resources from all packages ``Resources/Public/`` folders reside. * *persistent* which is the collection using the ``Storage`` and ``Target`` described in the respective section above to store any ``PersistentResource`` contents by default. Any new ``PersistentResource`` you create will end up in this storage if not set differently. Package Resources ================= Flow packages may provide any amount of static resources. They might be images, stylesheets, javascripts, templates or any other file which is used within the application or published to the web. Static resources may either be public or private: * *public resources* are represented by the ``static`` ``Collection`` described above and published to a web accessible path. * *private resources* are not published by default. They can either be used internally (for example as templates) or published with certain access restrictions. Whether a static package resource is public or private is determined by its parent directory. For a package *Acme.Demo* the public resources reside in a folder called ``Acme.Demo/Resources/Public/`` while the private resources are stored in ``Acme.Demo/Resources/Private/``. The directory structure below *Public* and *Private* is up to you but there are some suggestions in the :doc:`chapter about package management `. Both private and public package resources are not represented by ``PersistentResource`` instances in the database. Persistent Resources ==================== Data which was uploaded by a user or generated by your application is called a *persistent resource*. Although these resources are usually stored as files, they are never referred to by their path and filename directly but are represented by ``PersistentResource`` instances. .. note:: It is important to completely ignore the fact that resources are stored as files somewhere – you should only deal with resource objects, this allows your application to scale by using remote resource storages. New persistent resources can be created by either importing or uploading a file. In either case the result is a new ``PersistentResource`` which can be attached to any other object. As soon as the ``PersistentResource`` is removed (can happen by cascade operations of related domain objects if you want) the file data is removed too if it is no longer needed by another ``PersistentResource``. Importing Resources ------------------- Importing resources is one way to create a new resource object. The ``ResourceManager`` provides a simple API method for this purpose: *Example: Importing a new resource* :: class ImageController { /** * @Flow\Inject * @var \Neos\Flow\ResourceManagement\ResourceManager */ protected $resourceManager; // ... more code here ... /** * Imports an image * * @param string $imagePathAndFilename * @return void */ public function importImageAction($imagePathAndFilename) { $newResource = $this->resourceManager->importResource($imagePathAndFilename); $newImage = new \Acme\Demo\Domain\Model\Image(); $newImage->setOriginalResource($newResource); $this->imageRepository->add($newImage); } } The ``ImageController`` in our example provides a method to import a new image. Because an image consists of more than just the image file (we need a title, caption, generate a thumbnail, ...) we created a whole new model representing an image. The imported resource is considered as the "original resource" of the image and the ``Image`` model could easily provide a "thumbnail resource" for a smaller version of the original. This is what happens in detail while executing the ``importImageAction`` method: #. The URI (in our case an absolute path and filename) is passed to the ``importResource()`` method which analyzes the file found at that location. #. The file is imported into Flow's persistent resources storage using the sha1 hash over the file content as its filename. If a file with exactly the same content is imported it will reuse the already stored file data. #. The ResourceManager returns a new ``PersistentResource`` which refers to the newly imported file. #. A new ``Image`` object is created and the resource is attached to it. #. The image is added to the ``ImageRepository`` to persist it. In order to delete a resource just disconnect the resource object from the persisted object, for example by unsetting ``originalResource`` in the ``Image`` object and call the ``deleteResource()`` method in the ResourceManager. The ``importResource()`` method also accepts stream resources instead of file URIs to fetch the content from and you can give the name of the resource ``Collection`` as second argument to define where to store your new resource. If you already have the new resource`s content available as a string you can use ``importResourceFromContent()`` to create a resource object from that. Resource Uploads ---------------- The second way to create new resources is uploading them via a POST request. Flow's MVC framework detects incoming file uploads and automatically converts them into ``PersistentResource`` instances. In order to persist an uploaded resource you only need to persist the resulting object. Consider the following Fluid template: .. code-block:: xml This form allows for submitting a new image which consists of an image title and the image resource (e.g. a JPEG file). The following controller can handle the submission of the above form:: class ImageController { /** * Creates a new image * * @param \Acme\Demo\Domain\Model\Image $newImage The new image * @return void */ public function createAction(\Acme\Demo\Domain\Model\Image $newImage) { $this->imageRepository->add($newImage); $this->forward('index'); } } Provided that the ``Image`` class has a ``$title`` and a ``$originalResource`` property and that they are accessible through ``setTitle()`` and ``setOriginalResource()`` respectively the above code will work just as expected:: use Doctrine\ORM\Mapping as ORM; class Image { /** * @var string */ protected $title; /** * @var \Neos\Flow\ResourceManagement\PersistentResource * @ORM\OneToOne */ protected $originalResource; /** * @param string $title * @return void */ public function setTitle($title) { $this->title = $title; } /** * @return string */ public function getTitle() { return $this->title; } /** * @param \Neos\Flow\ResourceManagement\PersistentResource $originalResource * @return void */ public function setOriginalResource(\Neos\Flow\ResourceManagement\PersistentResource $originalResource) { $this->originalResource = $originalResource; } /** * @return \Neos\Flow\ResourceManagement\PersistentResource */ public function getOriginalResource() { return $this->originalResource; } } All resources are imported into the default *persistent* ``Collection`` if nothing else was configured. You can either set an alternative collection name in the template. .. code-block:: xml Or you can define it in your property mapping configuration like this:: $propertyMappingConfiguration ->forProperty('originalResource') ->setTypeConverterOption( \Neos\Flow\ResourceManagement\ResourceTypeConverter::class, \Neos\Flow\ResourceManagement\ResourceTypeConverter::CONFIGURATION_COLLECTION_NAME, 'images' ); Both variants would import the uploaded resource into a collection named *images*. All import methods in the ``ResourceManager`` described above allow setting the collection as well. .. tip:: If you want to see the internals of file uploads you can check the ``ResourceTypeConverter`` code. Accessing Resources =================== There are multiple ways of accessing your resource`s data depending on what you want to do. Either you need a web accessible URI to a resource to display or link to it or you need the raw data to process it further (like image manipulation for example). To provide URIs your resources have to be published. For newly created ``PersistentResource`` objects this happens automatically. Package resources have to be published at least once by running the ``resource:publish`` command: .. code-block:: none path$ ./flow resource:publish This will publish all collections, you can also just publish the *static* ``Collection`` by using the ``--collection`` argument. .. admonition:: Why Flow uses symbolic links by default Publishing resources basically means copying files from the ``Storage`` location to the ``Target``. In the default configuration Flow instead creates symbolic links, making the resources consume less disk space and work faster. By changing the ``Target`` configuration you can change this. Package Resources ----------------- Static resources (provided by packages) need to be published by the ``resource:publish`` command. If you do not change the default configuration the whole ``Resources/Public/`` folder is symlinked, which means you probably never need to publish again. If you configure some other ``Target`` make sure to publish the *static* collection whenever your package resources change. To get the URI to a published package resource you can use the ``getPublicPersistentResourceUri()`` method in the ``ResourceManager`` like this: .. code-block:: php $resourceUri = $this->resourceManager->getPublicPackageResourceUri('Acme.Demo', 'Images/Icons/FooIcon.png'); The same can be done in Fluid templates by using the the built-in resource ViewHelper: .. code-block:: html Note that the ``package`` parameter is optional and defaults to the package containing the currently active controller. .. warning:: Although it might be a tempting shortcut, never refer to the resource files directly through a URL like ``_Resources/Static/Packages/Acme.Demo/Images/Icons/FooIcon.png`` because you can't really rely on this path. Always use the resource view helper instead. Persistent Resources -------------------- Persistent resources are published on creation to the configured ``Target``. To get the URI for it you can rely on the ``ResourceManager`` and use the ``getPublicPersistentResourceUri`` method with your resource object:: $resourceUri = $this->resourceManager->getPublicPersistentResourceUri($image->getOriginalResource()); Again in a Fluid template the resource ViewHelper generates the URI for you: .. code-block:: html A persistent resource published to the default ``Target`` is accessible through a web URI like ``http://example.local/_Resources/Persistent/107bed85ba5e9bae0edbae879bbc2c26d72033ab/your_filename.jpg``. One advantage of using the sha1 hash of the resource content as part of the path is that once the resource changes it gets a new path and is displayed correctly regardless of the cache settings in the user's web browser. If you need to access a resource`s data directly in your code you can aquire a stream via the ``getStream()`` method of the ``PersistentResource``. If a stream is not enough and you need a file path to work with the ``createTemporaryLocalCopy()`` will return one for you. .. warning:: The file in the path returned by ``createTemporaryLocalCopy()`` is just valid for the current request and also just for reading. You should neither delete nor write to this temporary file. Also don't store this path. Resource Stream Wrapper ======================= Static resources are often used by packages internally. Typical use cases are templates, XML, YAML or other data files and images for further processing. You might be tempted to refer to these files by using one of the ``FLOW_PATH_*`` constants or by creating a path relative to your package. A much better and more convenient way is using Flow's built-in package resources stream wrapper. The following example reads the content of the file ``Acme.Demo/Resources/Private/Templates/SomeTemplate.html`` into a variable: *Example: Accessing static resources* :: $template = file_get_contents( 'resource://Acme.Demo/Private/Templates/SomeTemplate.html' ); Some situations might require access to persistent resources. The resource stream wrapper also supports this. To use this feature, just pass the resource hash: *Example: Accessing persisted resources* :: $imageFile = file_get_contents('resource://' . $resource->getSha1()); You are encouraged to use this stream wrapper wherever you need to access a static or persistent resource in your PHP code. Publishing to a Content Delivery Network (CDN) ============================================== Flow can publish resources to Content Delivery Networks or other remote services by using specialized connectors. First you need to install your desired connector (a third-party package which usually can be obtained through packagist.org9 configure it according to its documentation (provide correct credentials etc). Once the connector package is in place, you add a new publishing target which uses that connect and assign this target to your collection. .. code-block:: yaml Neos: Flow: resource: collections: persistent: target: 'cloudFrontPersistentResourcesTarget' targets: cloudFrontPersistentResourcesTarget: target: 'Flownative\Aws\S3\S3Target' targetOptions: bucket: 'media.example.com' keyPrefix: '/' baseUri: 'https://abc123def456.cloudfront.net/' Since the new publishing target will be empty initially, you need to publish your assets to the new target by using the ``resource:publish`` command: .. code-block:: none path$ ./flow resource:publish This command will upload your files to the target and use the calculated remote URL for all your assets from now on. Switching the storage of a collection (move to CDN) =================================================== If you want to migrate from your default local filesystem storage to a remote storage, you need to copy all your existing persistent resources to that new storage and use that storage afterwards by default. You start by adding a new storage with the desired driver that connects the resource management to your CDN. As you might want also want to serve your assets by the remote storage system, you also add a target that contains your published resources (as with local storage this can't be the same as the storage). .. code-block:: yaml Neos: Flow: resource: storages: s3PersistentResourcesStorage: storage: 'Flownative\Aws\S3\S3Storage' storageOptions: bucket: 'storage.example.com' keyPrefix: 'my/assets/' targets: s3PersistentResourcesTarget: target: 'Flownative\Aws\S3\S3Target' targetOptions: bucket: 'media.example.com' keyPrefix: '/' baseUri: 'https://abc123def456.cloudfront.net/' In order to copy the resources to the new storage we need a temporary collection that uses the storage and the new publication target. .. code-block:: yaml Neos: Flow: resource: collections: tmpNewCollection: storage: 's3PersistentResourcesStorage' target: 's3PersistentResourcesTarget' Now you can use the ``resource:copy`` command: .. code-block:: none path$ ./flow resource:copy --publish persistent tmpNewCollection This will copy all your files from your current storage (local filesystem) to the new remote storage. The ``--publish`` flag means that this command also publishes all the resources to the new target, and you have the same state on your current storage and publication target as on the new one. Now you can overwrite your old collection configuration and remove the temporary one: .. code-block:: yaml Neos: Flow: resource: collections: persistent: storage: 's3PersistentResourcesStorage' target: 's3PersistentResourcesTarget' Clear caches and you're done.