Technology

How Smesh works

Smesh’s ‘big data’ systems capture data from many different sources, perform filtering and analysis work on the data, and then make that data available for creative representation. Whether that’s on screen, on a mobile phone, a campaign site, an ad or an app.

Smesh won’t try to shorehorn you into a ‘one-size-fits-all’ solution or API, instead we specialise in bespoke customisation of our underlying platform for the specific needs of your project and domain.

Filtering & analysis

Smesh can perform many layers of automated filtering and analysis ‘out of the box’, all of which can be customised to meet the needs of particular projects:

  • Sentiment analysis
  • Profanity filtering
  • ‘Hero’ or ‘feature’ post selection
  • Topic extraction (associated term & phrase groups)
  • Over/under represented term analysis
  • Trending term & phrase detection
  • Higher order data analysis (e.g. trends in rates of change)

API access

Smesh uses APIs heavily as part of its internal infrastructure, to keep components independent of one-another. API-based access can be provided to Smesh clients for all functionality, within boundaries defined by 3rd party terms of service (i.e. contraints of using data from services like Twitter, Facebook and Instagram).

Moderation

Smesh supports a range of moderation options, including pre-moderation (nothing exposed unless it is approved) and post-moderation (content is exposed automatically, but can be removed easily).

Smesh’s projects often involve different layers of content with different approaches to moderation at each layer. For example, numbers and visualisation about social content (rather than the content itself) may be entirely automated and real-time. Social content that is exposed to users (e.g. the text of tweets or Instagram photos) but which changes rapidly may be post-moderated (as the content isn’t visible for very long). Content that is featured prominently or available for longer periods of time often warrants pre-moderation.

Performance and reliability

Smesh’s systems can be deployed in highly scalable and fault-tolerant architectures. As a minimum, we usually deploy a small cluster of load-balanced virtual hosts for any project, with frequent backups and automated failover to keep things running if a server dies.

We use various layers of high-performance caching systems, both on our own servers and using 3rd party services, to provide for very high load requirements, such as serving data to ads with millions of daily impressions, or mobile apps that refresh their data very frequently.

Multiple languages

Smesh’s systems have been used to power campaign content in various languages in addition to English, including Italian and French, as well as languages with different character sets such as Hindi, Mandarin Chinese and Japanese.

Open Source

Smesh takes pride in drawing on the best Open Source tools available rather than trying to build ‘proprietary’ systems end to end. From server management, monitoring and deployment through to ‘big data tools’ like Hadoop, MongoDB and Redis, we use existing high-performance tools with strong community support behind them. Smesh adds its own value through an architecture combining these components in a very flexible way, experience in working with and managing high-volume social data, performing complex analysis, and knowing how to build very cool bespoke things very quickly.