scala - Best approach to ingest Streaming Data in Lagom Microservice -


i creating streaming analytics application in each analytics/functionality implement microservice analytics can able use in different project later.

i using lagom creating microservice. new in lagom that's why came across doubts.

i don't understand best approach post stream of data (coming multiple sensors) microservice , microservice publish data kafka topic.

  1. does lagom feature of stream message in service description servicecall[ source[string, notused], source[string, notused]] right approach post streams of data (big data) hundreds of wifi sensors? have tenancy deal huge amount of data streams receiving near real time (=5 sec)?

  2. secondaly, while publish data kafka topics why have implement persistent entity (recommended lagom)? because kafka guarantees at-least-once delivery of message

my application not crud application, support process streaming data.

  1. lagom's streaming calls use websockets. it's built on play's websocket support, can scale millions of connected clients. wouldn't call hundreds of wifi sensors huge amount of data, lagom should handle it, , lagom can scaled horizontally, if processing you're doing heavy, can spread processing across many nodes.

  2. publishing incoming websocket stream kafka not supported in lagom. while kafka guarantee @ least once once message published kafka, there no such guarantees when getting message kafka in first instance. example, if side effect, such update database, publish message, there's no guarantee if application crashes between when database updated, , when message published kafka, that message published kafka (in fact won't be, message lost). why lagom encourages database event streams published kafka, since publishing event log in way guarantee database operation needs sent kafka happen @ least once. however, if you're not doing side effects, sounds you're not, might not relevant you. recommend in case use akka-streams-kafka (what lagom's kafka support built on) directly.

i've raised issue referencing use case here.


Comments

Post a Comment

Popular posts from this blog

commonjs - How to write a typescript definition file for a node module that exports a function? -

consider, toml node module can use: // toml.d.ts declare module toml { export function parse(value:string):any; } declare module "toml" { export = toml; } then: /// <reference path="../../../../../defs/toml/toml.d.ts"/> import toml = require('toml'); toml.parse(...); however, node module export single function, such 'glob' ( https://github.com/isaacs/node-glob ). the node usage of module be: var glob = require("glob") glob("*.js", {}, callback(err, files) { ... }); you'd naively expect this: // glob.d.ts declare function globs(paths:string, options:any, callback:{(err:any, files:string[]):void; ...but because typescripts 'import' semantics bit strange, seems can use 'import .. = require()' statement alias modules . trying call: /// <reference path="../../../../../defs/glob/glob.d.ts"/> import blog = require('glob'); results in: error ts2072: mod
Read more

openid - Okta: Failed to get authorization code through API call -

i'm integrating okta own idp server using okta's api. i'm implementing authorization code flow following steps below: in own server, use /api/v1/authn endpoint sessiontoken. use sessiontoken obtain authorization calling endpoint: /oauth2/v1/authorize?client_id=" + clientid + "&sessiontoken=" + sessiontoken + "&response_type=code&response_mode=query&scope=openid&redirect_uri=" + redirecturl + "&state=evanyang&nonce=" it's supposed return response status code 302 , location header containing redirect url code value. however, keep getting response status code 200 , without location header, html body saying "you using unsupported browser." , "javascript disabled on browser." according api documentation: http://developer.okta.com/docs/api/resources/oidc.html#authentication-request , sessiontoken parameter sufficient this: an okta one-time sessiontoken. allows api-based
Read more

thorough guide for profiling racket code -

did googling ("racket profiling", "racket measure performance"), didn't find , there no examples in docs. did google "profile" search on htdp - no luck. (profile (f ...)) output isn't obvious other small snippets. ideally want python's python -m cprofile usage examples. when search "python profiling", both duckduckgo , google give top result: 26.4. python profilers . (although scanned quickly, seems more of reference "thorough usage guide" examples. if had else in mind, perhaps please link that?) the equivalent racket documentation be: profile: statistical profiler . a usage example: #lang racket (require profile) (profile-thunk (thunk (function-to-profile arg0 arg1) )) here (thunk e) convenience (lambda () e) . bigger example: #lang racket (module mod racket (provide f) (define (f) (for/list ([i 10000]) i))) (require (prefix-in mod: 'mod)) (define (f) (for ([i 10000
Read more