Migrate from KissMetrics to Customer.io

Limetree, like a lot of companies out there, rely on KissMetrics to get insights on their users. We track lots of events and try to make sense out of them.

With all this data there is still on flaw. We can easily segment our customers and create amazing charts, but we cannot reach them. KissMetrics does not help here; their data is fairly close and there is no API available.

I want to email my customers that have used my service for a few weeks and then stopped. Or how about those that only send pictures, but no videos. I want to tell them that videos is an amazing part of our product.

How to migrate to Customer.io

We are not ditching KissMetrics but Customer.io relies on it’s own event system so I needed a way to import my old metrics. I used the export service from KissMetrics which will dump hundreds of JSON files to a S3 bucket of your choice. It will take some hours on the first time but after that they will keep the bucket updated every few hours.

Step 0: KissMetrics to Customer.io Migration Class

Step 1: Export the KissMetrics data

This step is simple and KissMetrics has a simple guide: Follow this steps and come back for the next step.

Step 2: Export your users

Before you start sending the events to Customer.io you need to create the customers – which is the same as users. Export your userbase to a CSV file. A simple dump from your database.

The required fields are: user id, email, name and creation date. You can add other properties you might find useful – probabily those you were already using on KissMetrics: type of plan or country.

23123, "John Doe", "[email protected]", "2012-11-23", "Portugal"
...

Step 3: Import your users

Feed each user to create_customer(id, email, createdAt, attributes)

This process will take a few minutes to hours depending on your userbase.

error_reporting(-1);

define('SITE_ID', 'xxxx');
define('API_KEY', 'yyyy');

require("kissmetrics-to-customerio.php");

$csvFile = file_get_contents('users-dump.csv');

foreach (explode("\n", $csvFile) as $row)
{
    $data       = str_getcsv($row);
    $id         = $data[0];
    $name       = $data[1];
    $email      = $data[2];
    $createdAt  = strtotime($data[3]);
    $country    = $data[4];

    $attributes = array('name' => $name, 'country' => $country);

    try {
        create_customer($id, $email, $createdAt, $attributes);
    } catch (Exception $ex) {
        echo $ex->getMessage();
    }
}

Step 4: Collect your events

After you imported all your users the S3 bucket should be filled with the juicy content of all your KissMetrics events. The number of files can be overwhelming. They export hundreds of small files each containing several one-liners with JSON; we will need to compact it into one to facilitate the import process. Example:

{"platform":"iphone","_n":"login","_p":"12312","_t":1352317082}
{"platform":"web","_n":"login","_p":"2221321","_t":1352316831}
{"platform":"web","_n":"login","_p":"112123","_t":1352317100}

First sync the S3 bucket with a local directory:

s3cmd sync s3://kissmetrics-export-bucket km-export

Compact all the JSON files into one:

cat km-export/*.json > km-data.json

Step 5: Import your events

Feed each event to track_event(userId, name, timestamp, attributes)

This process will take a few hours depending on the volume of events.

define('SITE_ID', 'xxxx');
define('API_KEY', 'yyyy');

require("kissmetrics-to-customerio.php");

$eventsFile = file_get_contents('km-data.json');

foreach (explode("\n", $eventsFile) as $row)
{
    $data       = json_decode($row, true);

    $userId      = $data['_p'];
    $name        = $data['_n'];
    $timestamp   = $data['_t'];

    unset($data['_p']);
    unset($data['_n']);
    unset($data['_t']);

    // After unsetting _p, _n and _t, the remaining are event properties
    $attributes = $data;

    try {
        track_event($userId, $name, $timestamp, $attributes);
    } catch (Exception $ex) {
        echo $ex->getMessage();
    }
}

Step 6: Wait

On this step you can grab a coffee, watch a movie, do some more coding or simply sleep. It’s gonna take a few hours but I guarantee you that will save you days or even weeks of work. The data your already have on KissMetrics is worth a lot.

Step 7: Extra tip

If you have a huge amount of data – a few months or even years – you can accelerate this process by importing only the last 30 days, for example. There are some use cases where you might need data from the Past but this timeframe worked for me.

Just add this piece of code:

$eventsSince = strtotime('-30 days');

foreach (explode("\n", $eventsFile) as $row)
{
    $data       = json_decode($row, true);

    $userId      = $data['_p'];
    $name        = $data['_n'];
    $timestamp   = $data['_t'];

    if ($timestamp < $eventsSince) continue;

    …
}

Conclusion

This guide may seem big but you can do this process in a few hours. If you have any doubt just tweet or send me an email: diogoepf at icloud dot com.

Links

ChillOut Everyday Player

ChillOut Everyday Player

Last night I launched a new player for the ChillOut Everyday page. It is very simple and stylish and you can find there dozens of good music non-stop. More players with other genres will be launched soon.

I won the Hackafone

Last weekend I went to OPorto, Portugal, to participate in the Hackafone: a mobile application contest where participants have to brainstorm, create an idea and produce it in 18 hours. The two winners would win a Samsung Galaxy Tab 10.1 and, above all, the app will be marketed by Vodafone in their app store and communication channels.

The event was hosted in the Geeks Lab of muchBeta, a web creative saas power house, located in Afurada, near the Douro River; p.s. great view!
The app was created with @ricardjorg and @luismvaz. We are going to finish it until the end of January 2012 and you can expect it in an Android, iPhone and iPad near you. Following is a sneak preview of the splash page;

Thank you muchBeta team, Vodafone for believing in new talents, the other hackthletes and my team.


Continue reading

Analytics with Redis

This is my experience using Redis.io as a datastore to support an analytics system. I originally used MySQL for the job but for some reasons it did not correspond to my expectations.

The objective for this system is to give hourly social metrics like page fans, page views, posts shares, from Facebook, but also how much likes, shares and tweets the posts from several websites received. This would enable, later, the visualization how page posts, that link to the website content, influence the ecosystem.

Continue reading

Working in Sanoma Media

Since May 2011 I am working on Sanoma Media, a subsidiary of Sanoma which operates in 20 european countries and employed over 15k last year.

I will be producing web applications mainly in Facebook and other social networks. More updates in the future.

 

Agenda cultural MyOut

Este é um novo projecto nacional. Criado por três amigos que estavam fartos de perder os seus concertos favoritos.

Pesquisa as tuas bandas favoritas faz “Seguir” e recebe notificações sempre que sejam agendados novos concertos. Faz do Myout.net a tua agenda.

  • Utilizando a pesquisa geral poderás encontrar facilmente eventos, espaços ou artistas
  • Filtra os eventos por cidade, preço ou data
  • Também podes ver os próximos eventos na vista semanal
  • Segue os teus artistas favoritos, espaços favoritos e utilizadores. Recebe notificações quando forem criados novos eventos
  • Recebe recomendações de eventos baseados nos teus gostos
  • Deixa o teu comentário em eventos e espaços
  • Escreve a tua opinião e classifica espaços e eventos interagindo com a comunidade MyOut
  • Faz login com o Facebook e terás automaticamente os teus artistas e espaços favoritos importados
  • Divulga os teus eventos e partilha (a criação é rápida e gratuita)
  • Promove o teu espaço: podes adicionar eventos, personalizar a página e atrair novos seguidores

 

http://www.myout.net

 

Video Conferencing with HTML5

Did you know that work is being done to enable videoconferencing from HTML5 applications? Ian Hickson has been doing work on the element in a separate draft to make this possible.

Read full article

Live video streaming with PHP

Picture this. We want to stream live video. In fact don’t need PHP. We only need a flash player (or HTML5) and our live feed. The problem appear when we need to offer some kind of security.

Read the full article

Sync Gmail Contacts and Google Calendar with Outlook

There are many tutorials out there but they are all too complicated or/and give you paid, complicated tools to do the job.

Both tools on this post are free, will sit on your tray and won’t bother you, while maintaining your contacts and calendar synced. I’m gonna show you the easiest way to do it.

Google Calendar Sync

google calendar

Download page: http://www.google.com/support/cal..wer=89955
(Direct link to download)

Installation

  1. Read through the Google Calendar Sync Terms of Service, and click I Agree.
  2. Follow through the Installation Options and click Install to finish the set-up process.
  3. Once Google Calendar Sync is installed on your computer, the Google Calendar Sync Settings window will appear:

Google Calendar SyncPrint

Just enter your email/password and click Save. If you don’t want to sync back to Google Calendar choose “1 way: Google Calendar to Microsoft Outlook”

You are now syncing Google Calendar with Outlook!

Continue reading

ToneMatrix

This is one piece of flash that will make you addicting in the first seconds. You just click and move your mouse over the board and, even if you don’t want, you’ll create a vicious melody.

To clear the board press space.
Play ToneMatrix now!