How do I get Akamai mPulse data into Google Cloud Storage?

Datastream 2 seems to support a ton of output options, but mPulse only supports S3?

How can I get mPulse into Google Cloud Storage?

Could I setup like a pub/sub endpoint or something?

1 Like

mPulse has a Query API for aggregated data. (you can query 13 months worth of data, same as in the UI)

The existing export function is mainly for archival purposes of the raw data (=a lot). S3 is currently the only option. From S3 you could push it to other locations of your choice.

Alternatively you can modify the mPulse library to not only send beacons to the mPulse collectors but also to a 2nd endpoint of your choice. As some data is filtered on the the mPulse collectors this can lead to 2 different data sets.

Hello and welcome to our community, @borgil . I hope you don’t mind that I’ve added the tags mpulse and datastream to your post for better discoverability. They weren’t available in this category before, so you may have already tried to add them yourself. :slightly_smiling_face:

1 Like

What is the export interval allowed? Had trouble finding that in the docs.

Can I have it export to S3 every minute for example?

You can not configure an export interval, this will be a continuous stream of logs arriving from different locations (mPulse collects beacons from several locations around the globe). With a few minutes (5-10) delay to allow mPulse to synchronize/sanitize the information.

Are you planning to use the data in actual data in mPulse (kept for 13 months)? Or will you use your own tools to graph the exported data?

I’d like to experiment with both, but having all the raw data, especially if there is a 10 min delay might not be necessary. Are there plans to support GCS? Or is there a log API where I could fetch the raw data myself?

I’m curious about this:

Alternatively you can modify the mPulse library to not only send beacons to the mPulse collectors but also to a 2nd endpoint of your choice

So instead of polling the API myself I can I have it push data to my own authenticated endpoint? For auth would oauth be supported?

Thanks!

1 Like

mPulse = Boomerang + Easy to use config layer + dashboards + scalable beacon collection

Boomerang is Open Source (maintained by Akamai)

You can tweak the Boomerang client JS and have it send 2 beacons. 1 to Akamai mPulse, 1 to whatever endpoint directly from the client.

Or depending on your goals with mPulse, use the Open Source Boomerang version and have it sent to a system of your choice.

Hey I’ll look into that!

Got another question if I may, does your terraform provider support configuring mpulse? Could I config S3 export and or other configuration for mpulse with it? I don’t see anything about it here: https://registry.terraform.io/providers/akamai/akamai/latest/docs/resources/resources

Also, have to admit my ignorance of the product suite, is mpulse a new thing or a soon to be sunsetted thing? I ask because relative to something like datastream I don’t see a lot of features wrt to this kind of administrative management.

No mPulse has no terraform registry yet.

Yes mPulse is a very active RUM solution, a big release was just done this week and several others are in the pipeline.

Yesterdays release