Slingshot
Slingshot is the SNS automation engine for EVO that powers search, preview, replication, and more. It provides task management for repetitive file operations on EVO and other systems, including cloud storage. In addition to indexing and proxy generation, user-created Slingshot Automations can perform actions on schedule, and Replication Jobs can copy and back up to and from remote SMB or cloud targets.
Slingshot is available as a component requiring initialization, and is not required for standard EVO operation, though it does provide for many additional features and added functionality, including ShareBrowser.
Initialization
Components are often started as a convenience during commissioning; however, there may be instances in which ShareBrowser/Slingshot are not initialized. If components are not initialized, clicking ShareBrowser or Slingshot in the interface will return “Scheduled backup for component databases is not configured”
Once configured, EVO will automatically generate backups of the ShareBrowser and Slingshot databases. This requires allocation of some storage (typically under 1 GB, depending on database and retention policy). Click to specify share for backup and restore of the database.
Locate the Component databases card at the Support page.
Click to edit and select a share to store database backups. This is often a share for which only an administrator or secondary administrator is granted permission (typically named “sys-management”).
Save and click the Slingshot menu item again to Initialize ShareBrowser Server and Slingshot.
Initializing requires a dedicated system share for the active automation engine. The share needs to be in the Reserved permission mode, recommended with at least 200 GiB available for system use.
If an existing database is to be restored, the previously exported backup may be uploaded or an existing backup on EVO may be selected. Click the dots at the Component Databases card to import an existing database.
Select “Upload file” to drag and drop or browse for the database backup file on your workstation, or select “Use restore location” to import from an existing backup location on EVO.
Tip
If there’s ever a need to deinitialize components, the option is found at the #/master-reset page.
Replications
Schedule replication operations between volumes or directories. Configure any needed external targets such as Amazon S3 or remote SMB shares, and then click the plus sign to set up a replication job.
Configure a job to copy new/changed folders and files from the selected source to the selected destination. Objects with the same path/name will be overwritten at the destination. File permissions, metadata, and other attributes may not be applicable for backup. Cloud-based transfers will incur bandwidth charges by your cloud storage provider. Bi-directional job schemes are not supported.
Copy/Replace vs Sync/Remove
Choose “Copy/Replace” if objects removed from the source location should NOT be automatically removed from the destination location.
Choose “Sync/Remove” if objects removed from the source location should be automatically removed from the destination location. Use “Sync/Remove” with caution to avoid accidentally deleting data at the destination!
More information is found in the Slingshot Replication (Sync/Copy) best practices and configuration guide: https://support.studionetworksolutions.com/hc/en-us/articles/360027021212-Replication-Sync-Copy-best-practices
Job name - Give the job a descriptive name. Note that spaces and other special characters are not allowed.
Method - The options presented here are Copy/Replace or Sync/Remove. Replication jobs default to Copy/Replace, which means content added to the source will be copied to the destination, and any content that’s been deleted from the source since the job was last run is left in place on the destination. If Sync/Remove is chosen, content added to the source is added to the destination, and if content has been deleted from the source since the last job was run, it will also be deleted from the destination.
Source - Select the source to be replicated. The default selection is for a local EVO share. It’s also possible to replicate a complete EVO logical disk, another SMB or FTP target, or a connected cloud location.
Share (or Logical Disk/Alias) - This field changes according to the Source selection. Select the share, disk or alias to use as a source for replication.
Path - Enter or BROWSE to a specific directory on the above selection if needed. If left blank, replication will work at the root (top level folder) of the above selection.
Destination - Select the target destination to which the source selection should be replicated.
Share (or Alias) - This field changes according to the Destination selection. Select the share or alias to use as a destination for the replication target.
Path - Enter or BROWSE to a specific directory on the above selection if needed. If left blank, replication will work at the root (top level folder) of the above selection.
Scheduling controls - Set the frequency for the replication job. This can be a manual job run on-demand, or a scheduled operation to run once or recurring at a defined interval. Select the Begin and End dates, and then choose the start day/time for the operation to run. The job will end on its own unless an End time is specifically defined. If the job is still running at that time, it will be paused until the next start interval.
Configure email settings on the Alerts page. If SMTP is configured, email notifications can be sent for job summary, job start, and changes to the job status (pause/resume).
Advanced settings
Filtering - Include logic for transferring files based on age, or configure (golang) regex pattern matches for inclusion/exclusion behavior. Choose whether or not to include the recycle bin (if present at source).
ShareBrowser - If included, Slingshot can ensure ShareBrowser metadata and/or proxy links are preserved. These metadata are not replicated with the file; rather, replicated content is reflected in the ShareBrowser database, allowing the database to be searched and or previewed even when the original files have been replicated to another system and may no longer be directly accessible.
Verification - By default, replication jobs will copy any file that is not yet found at the destination, or any file is newer at the source than at the destination. Files at the destination that are newer than the unchanged source files (previously replicated) will be ignored.
Skip files with a modified time newer than the source file - Enabled by default to only update what’s needed. This can be disabled to ensure all files are replicated, overwriting any that already exist at the destination.
Detect file modification based on time/size/content rather than time/size - Disabled by default. While rare, it is possible for differing source and destination files to share time/size statistics. Enabling additional detection for content modification can protect against false matches; note that the added processing will impact replication time.
Verify file integrity after transfer… - Disabled by default. The default calculations for data comparison are sufficient in most scenarios, while some environments may require bit by bit verification. The added calculations will greatly increase the time required for the replication job to run, though this does provide a record of proof that verification was completed. The Report value can be set to show Only differences, or All files.
Job handling - Files are replicated in parallel by default (multiple actions). This can be changed to serial to ensure single-stream writes. A Transfer Summary is automatically provided at job’s end, while it’s also possible to save more detailed logs for the job at the destination.
Log level - Replication jobs will provide a standard level of logging to EVO logs by default, or at the destination if configured. This can be set to No logging or Full logging for the most verbose output.
Warning
Slingshot replication jobs do not consider ACLs (file-level permissions), so user visibility and access permission for replicated content needs to be managed at the target destination.
Click on an existing Replication job to expose/hide options.
Disable/Enable - Toggle the status for the replication job
Edit - Make changes to a disabled replication job (enabled jobs may not be edited)
Run Now - Ignore schedule and run on demand
Show Last Job Summary - View summary of the last iteration of the selected replication job.
Show Instances - View more detailed information according to Instances > Processed files > Processed tasks
Index/Proxy
Indexer name
The indexer name generally reflects the host name. Setting a custom indexer can be especially useful in a multiple EVO environment. Click the pencil to edit the name.
Multiple EVO environment
ShareBrowser clients will need to log in to a ShareBrowser database, and if multiple EVOs exist in the environment, each one could have its own database, so it’s useful to select one EVO to store the primary database, and configure other EVOs to index to that central server. Click the pencil and then toggle “Push metadata to a central EVO” on, and new options will appear.
HTTPS - All EVOs involved will need to have https configured.
Central EVO IP address - Enter the IP address for the central EVO that holds the primary ShareBrowser database.
Port - This is 80 for http and 443 for https by default, though a custom port can be specified.
Proxy destination
The Slingshot engine can be configured to automatically create proxy files from original media. To configure proxy generation, a share needs to be designated to house the created proxies. This share is typically named something like “Proxies” and is for use by the ShareBrowser database only, so no users should need permissions.
Share to store proxy - Select a share from the dropdown menu to dedicate to proxy storage.
Relative path - Specify a folder if needed; otherwise, the root of the share will be used.
Preview/Proxy formats
Depending on file type, the proxy output could be a thumbnail, audio, or video preview. The default behavior should ensure each supported file is transcoded according to its type, and this section can be used to specify global formats for preview/proxy types. This could be used to ensure only thumbnails are generated, for example, rather than thumbnails and video previews. The default ShareBrowser preview (what’s viewed in ShareBrowser without direct access to the volume) selection is “Media proxy”, which is a lightweight (H.264) version of the original file. The “Edit-ready proxy format” is Pro Res by default, which is generally still going to be smaller than the original file, while providing more options for offline/proxy editing than a standard ShareBrowser preview proxy would.
ShareBrowser preview format - Media proxy by default (H.264). Use the dropdown menu to select an alternative if needed.
Edit-ready proxy format - Edit proxy by default (Apple Pro Res). Use the dropdown menu to select an alternative if needed.
CPU limit
EVO is designed with editors in mind, and the primary goal is to ensure users have the resources available to work simultaneously with shared media, while EVO may also be tasked with numerous automation jobs. One goal is to find a balance between human users and automated jobs, often configured to run after-hours when users are not working on the system. To ensure users have the needed resources available, the processing power dedicated to automations such as indexing, proxy generation, and replications is limited to 25% by default. If resource management and scheduling is considered, these values can be adjusted to tune the task processing specifically according to the needs of the environment.
Preview CPU load limit - By default, 25% CPU is allocated for generation of ShareBrowser previews of indexed content. This can alternatively be set to 50% or 75%.
Edit CPU load limit - By default, 25% CPU is allocated for generation of edit-ready proxies of indexed content. This can alternatively be set to 50% or 75%.
Enable proxy generation
The list of shares available for proxy generation should be automatically populated with any shares for which proxy generation is possible. Once storage has been configured to contain the proxy output, shares can be configured for automatic preview and/or edit-ready proxy generation.
Select a share to bring up its configuration options.
Enable ShareBrowser Preview Proxies - Click to enable. The light should turn red, with the option to again disable.
Enable Edit-ready Proxies - Click to enable. The light should turn red, with the option to again disable.
Exclude Extensions - Define any file types for which you would not like proxy generation to be attempted. Enter an extension such as “.mp4” and click ADD and then SAVE to instruct Slingshot to ignore all files with the matching extension.
Exclude Folders - Specify or browse to any folder(s) you’d like Slingshot to ignore for proxy generation and click ADD and SAVE.
Enable Prevew Proxies for all media that will be seen in ShareBrowser. Care should be taken when choosing a selection for Edit-Ready Proxies, which may require a significant amount of time to generate and disk space to store. In some cases, it’s possible the resulting edit-ready proxy may be larger than the original file. Choosing Manual proxy generation to allow ShareBrowser users to request edit-ready proxies as needed is a safe approach to ensure only intended media is transcoded at a higher quality or custom preset.
Schedule
In order to ensure users have the resources they need, all proxy generation is paused during the day by default (7:00 AM to 10:00 PM), and ShareBrowser users are not allowed to request proxies on demand. Click the pencil to edit these choices.
Pause all automatic proxy creation during these hours - On by default. Disabling means Slingshot will work around the clock if needed, and users may generated proxies on demand.
Pause time - With pause enabled, specify the time that the proxy generation queue should stop processing new items.
Resume time - With pause enabled, specify the time that proxy generation may resume.
Generate on-demand proxies from ShareBrowser users during these hours - When enabled, allows users to make custom proxy generation requests even when Slingshot proxy generation is otherwise paused.
Default exclusions
Some file types are expectedly unsupported for proxy generation, and excluding them can increase efficiency for the automation tasks. For example, a video preview cannot be generated from a compressed .zip file, so excluding all zip files from consideration can save time. This list is pre-populated with some default file types that should be skipped by the transcode engine. Click the pencil to add or remove any extensions for exclusion.
Automations
This section exposes automations, which may be user-defined, indexing or proxy generation tasks, or AI/ML functions. Select the type from the dropdown menu to expose its options.
User automations
Custom tasks or series of tasks may be scheduled to run on specific folders, including file copies, moves, renames, transcodes, and more. On-demand user-specific and recurring global tasks can be defined to automate a workflow based on watch folders. Click the plus sign to create a custom folder-level automation.
Automation name - Give your automation a descriptive name.
Recipients - If SMTP is configured, email notifications can be sent informing of automation completion.
Share to watch - Select the share on which the automation action should be taken.
Folder to watch - Specify a watchfolder for the automation task. If left blank, the root of the share will be used. Note that this is not recursive, so the selected folder’s subdirectories are not considered by the automation.
Launch - If the automation should run on a defined interval, choose Scheduled (default) and select the frequency (minute, hour, day) and start time for the automation.
Query - Root of folder only. Select whether the automation should query By files extension(s) (e.g. txt or * for all files) or Regular expression (e.g. d+.*).
Note
Preservation of filesystem hierarchy gets complex very quickly for automations, especially when source and destination file structures and permissions may differ. Custom user automations are therefore run on a single folder’s root level.
Save the automation when done and then click it again and choose EDIT AUTOMATION to add your task(s). Tasks can be stacked, so it’s possible to perform many chained actions within the same automation.
Warning
Automations have the power to change data. Always practice the automation logic on test files/folders before putting an automation into production.
Click Create Task to expose the options.
Task - A dropdown allows for selection from the list: Copy, Transcode, Delete, Move to today’s processed folder, Append automation data to file name, AI/ML Autotagging
The remaining settings change depending on task selection.
Copy
Source - Select Original file/folder or previous task result to use the output from a previous task from the same automation job.
Log level - Choose from No logging (default), Standard or Full logging.
Options - Move will move the file rather than create a copy. Replace will overwrite an existing instance of the output.
Scheme - Select Local (default) for a local file system operation, or Alias to select from an existing alias.
User/Alias - If the automation will work with a target requiring authentication, enter the relevant alias or user owner of the automation. This field can be left blank otherwise.
Relative path - Define or browse to the intended source.
Advanced settings
Create folder - When Create target directory is enabled (default), a new directory will be created at the destination if needed.
Speed limit - Enable to define a transfer speed limit in KB/s.
Transcode
Source - Select Original file/folder or previous task result to use the output from a previous task from the same automation job.
Log level - Choose from No logging (default), Standard or Full logging.
Preset - Default will perform the default transcode for each file type (preview proxy, thumbnail, etc.), and more actions can be chosen from the list, including differing thumbnail sizes, Media proxy, Edit proxy, AI/ML Autotagging, or Audio proxy.
Reaction on unknown streams - If the task encounters an issue, the behavior can be set to Fail (default), Ignore, or Copy. If set to Fail, several attempts are made. If set to Ignore, the problematic file is skipped. If set to Copy, the original is copied (rather than transcoded).
Share name - Specify or browse to the share on which the task is to take place.
Destination path - Define or browse to the destination directory for the transcode output.
Delete
The Delete option can be useful for removing original files once automation tasks have taken place.
Source - Select Original file/folder or previous task result to use the output from a previous task from the same automation job.
Log level - Choose from No logging (default), Standard or Full logging.
Move to today’s processed folder
This option can be useful for stacked tasks and can prevent unintended recursion for the working directory.
Source - Select Original file/folder or previous task result to use the output from a previous task from the same automation job.
Log level - Choose from No logging (default), Standard or Full logging. A folder named ‘automation ID + ‘’-’’ + yyyy-mm-dd’ is automatically created in the watched folder the first time this task fires on a new day.
Append automation data to file name
This option can be useful as an intermediary step in an automation.
Source - Select Original file/folder or previous task result to use the output from a previous task from the same automation job.
Log level - Choose from No logging (default), Standard or Full logging. Append automation ID + ‘’-’’ + yyyy-mm-dd-hh-mm-ss to file name
AI/ML Autotagging
Source - Select Original file/folder or previous task result to use the output from a previous task from the same automation job.
Log level - Choose from No logging (default), Standard or Full logging.
Access Code - Optionally define a code to authorize use of the cloud AI service.
Indexing tasks
Selecting indexing tasks from the dropdown menu shows scheduled automations for indexing and proxy generation. Volumes are automatically indexed at 10:00 PM daily by default (the schedule can be adjusted in the workstation ShareBrowser Admin application). The proxy generation service runs hourly, based on what’s already been indexed.
Click on en existing index/proxy automation to bring up its ACTIONS and DETAILS.
Run Now/Stop Run - When the job is idle, the option is presented to run it manually. When running, the option to stop it is presented.
Pause - A job can be put on hold without being stopped altogether, for example to free up resources temporarily.
Show Instances - This provides details about the active/previous job, including dynamically updating columns for estimated file counts (Total, Succeed, Failed, Unprocessed). Clicking on a row brings up Processed files, where again clicking on an ID row brings up Processed tasks.
Note
When configured, the proxy generation service runs hourly based on what’s been indexed. This is different from an on-demand (ad hoc) proxy request by a ShareBrowser user, in which case the file is indexed and transcoded (if configured and allowed).
Click an automation to bring up its ACTIONS and DETAILS.
Enable/Disable - Click to change the status for the automation.
Edit Automation - Make changes to an existing automation.
Run Now/Stop Run - When the job is idle, the option is presented to run it manually. When running, the option to stop it is presented.
Pause - A job can be put on hold without being stopped altogether, for example to free up resources temporarily.
Show Instances - This provides details about the active/previous job, including dynamically updating columns for estimated file counts (Total, Succeed, Failed, Unprocessed). Clicking on a row brings up Processed files, where again clicking on an ID row brings up Processed tasks.
AI/ML Records
If configured for cloud-based autotagging, details from those user-requested automations are displayed here.
Settings
Aliases
Aliases store credentials to define a location used by automations, so they can correctly authenticate to various targets when run. Consider whether the alias should be global or personal (per-user). Global aliases are generally created by the EVO admin user. Personal aliases are created by the alias owner after logging in to EVO’s web interface as that user (rather than admin), and are available only to that user.
Name - Enter a name for the alias to distinguish it from any other aliases.
Schema - Select the authentication type: SMB, FTP, Amazon S3, Azure, Dropbox, or Google Cloud Storage
The remaining settings change depending on schema selection.
Local:
User - Define a user that has access to the share.
Share name - Specify or browse to the share needing authentication.
Relative path - Define or browse to the intended source.
Private - If logged in as the alias owner, should be enabled (default). Personal aliases define transport destinations for a specific ShareBrowser user, and can be invoked via API or GUI. If a user owns an alias, a context menu is added to their ShareBrowser Client, which appears when right-clicking files in the Desktop Client. Clicking this will copy the selected content to the location defined in the alias. See the ShareBrowser guide for more information: https://studio-network-solutions-sharebrowser-docs.readthedocs-hosted.com/en/latest/
SMB
This establishes a connection separate from a mount created at the Shares page. Existing Remote shares may also be used in automations, while there may be scenarios where a discrete SMB connection is preferred for an automation.
IP/hostname - Enter the address or name for the remote SMB server.
Share name - Define the path to the share
User - Enter a valid user name for the remote server
Passsword - Enter the user password
Relative path - Define the path to the destination on the share if needed.
Private - If logged in as the alias owner, this should be enabled (default). Personal aliases define transport destinations for a specific ShareBrowser user, and can be invoked via API or GUI. If a user owns an alias, a context menu is added to their ShareBrowser Client, which appears when right-clicking files in the Desktop Client. Clicking this will copy the selected content to the location defined in the alias. See the ShareBrowser guide for more information.
FTP
IP/hostname - Enter the address or name for the remote SMB server.
User dir is root - Disabled by default. Enable if needed (if user directory is root).
Passive - Enabled by default, which can help prevent firewall interference. Disable if the FTP server denies PASV.
User - Enter a valid user name for the remote server
Passsword - Enter the user password
Relative path - Define the path to the destination on the share if needed.
Private - If logged in as the alias owner, this should be enabled (default). Personal aliases define transport destinations for a specific ShareBrowser user, and can be invoked via API or GUI. If a user owns an alias, a context menu is added to their ShareBrowser Client, which appears when right-clicking files in the Desktop Client. Clicking this will copy the selected content to the location defined in the alias. See the ShareBrowser guide for more information: https://studio-network-solutions-sharebrowser-docs.readthedocs-hosted.com/en/latest/
Amazon S3, Azure, Dropbox, and Google Cloud Storage
See the Slingshot Replication (Sync/Copy) best practices and configuration guide at https://support.studionetworksolutions.com/hc/en-us/articles/360027021212-Replication-Sync-Copy-best-practices
Note
Some AWS services may region-specific. Check the AWS console to ensure your bucket is in a region that provides AI/ML functionality.
Autotagging machine
When artificial intelligence/machine learning services are configured, ShareBrowser users can upload media for machine analysis and automatic content tagging.
Prior to configuration on EVO, some other steps need to be taken to add automatic tagging using AI/ML services:
Create an AWS account with Amazon. Take a look at their billing costs for the features you intend to use.
Enable S3 and Rekognition services.
Create an IAM user with permissions for AmazonRekognitionFullAccess and AmazonS3FullAccess.
Create an S3 bucket and make sure that the created user has permissions to access this bucket.
Generate the access key (Access key ID and Secret Access Key) on the user’s page of Amazon Services.
Region/Custom host - Select the AWS region for your bucket or enter the custom host information.
Bucket - Enter the S3 bucket name.
Access key - Enter the access key provided by AWS.
Secret key - Enter the secret key provided by AWS.
S3 relative path - Define a path if needed
Enable transcoding - By default, a temporary proxy of the original file is created for upload to Rekognition when a ShareBrowser user clicks Analyze. Sending a smaller version of the original file can substantially reduce bandwidth usage for the cloud service. The Preset determines the quality (and therefore, size) of the proxy that’s generated. Higher-resolution proxies may yield more accurate tagging results at the cost of increased bandwidth. A temporary folder needs to be designated for proxy staging during the operation when this setting is enabled.
Preset - The AI/ML Autotagging preset is used by default, which is designed to be as lightweight as possible for metered connections, though it’s possible to select from any available preset or click EDIT to change specific values for the selected preset.
Share name - If transcoding is enabled, select a share to use for temporary transcode storage.
Temporary folder - Specify or browse to a directory on the share to house the temporary proxy for upload.
Access code - Optionally create a custom access code to protect against unwanted bandwidth consumption. This will require users to enter the access code in order to submit media from the ShareBrowser web app for autotagging.
Presets
Current transcoding presets are displayed, with the option to create new custom presets or to edit an existing global preset value. Take care to note the original settings if experimenting with adjustment of a global default preset. Custom preset construction is outside the scope of this document.
Replication logs
A transfer summary report is created for each replication job, while more details about the replication are logged. By default, this is stored on EVO’s OS disk. It’s possible to instead send the log output to a NAS share, which could then be mounted and reviewed by a local administrator, or simply preserved for a history of replication instances (logs will rotate on OS disk).
Use OS disk - Default. Logs will rotate automatically, with recent replication logs included in EVO logs.
Use NAS share - Select a share and folder location; the “evo-replication-logs” folder is suggested, with the option to use the root of the share or create a new destination directory.