cuda - Amdahl's law and GPU -


i have couple of doubts regarding application of amdahl's law respect gpus. instance, have kernel code have launched number of threads, n. so,in amdahl's law number of processors n right? also, cuda programming using large number of threads, safe me assume amdahl's law reduced 1/(1-p) wherein p stands parallel code? thanks

for instance, have kernel code have launched number of threads, n. so,in amdahl's law number of processors n right?

not exactly. gpu not have many physical cores (k) number of threads can launch (n) (usually, k around 103, n in range 104 -- 106). however, significant portion of kernel time (usually) spend waiting data read/written from/to global memory, 1 core can seamlessly handle several threads. way device can handle n0 threads without them interfering each other, n0 several times bigger k, depends upon kernel function.

in opinion, best way determine n0 experimentally measure performance of application , use data fit parameters of amdahl's law :)

also, cuda programming using large number of threads, safe me assume amdahl's law reduced 1/(1-p) wherein p stands parallel code?

this assumption means neglect time parallel part of code (it executed infinitely fast) , consider time serial part.

e.g. if compute sum of 2 100-element vectors on gpu, initializing of device, data copying, kernel launch overhead etc (serial part) takes more time kernel execution (parallel part). however, not true.

also, individual gpu core not have same performance cpu core, should scaling, making amdah'l law 1 / [(1-p) + k*p/n] (at it's simplest, k = frequency(cpu) / frequency(gpu), k increased more take account architectural differences, cpu core having simd block).

i argue against literally applying amdahl's law real systems. sure, shows general trend, not grasp non-trivial processes.

first, amdahl's law assumes given infinite number of cores parallel part executed instantly. assumption not true (though might pretty accurate). if calculate sum of 2 vectors, can't compute faster takes add 2 bytes. 1 can neglect "quanta", or include in serial portion of algorithm, "breaks" idea.

how correctly estimate in amdahl's law effect of barrier synchronization, critical section, atomic operations etc. is, best of knowledge, unresolved mystery. such operations belong parallel part, walltime of execution @ best independent of number of threads and, @ worst, positively dependent.

simple example: broadcasting time between computational nodes in cpu cluster scales o(log n). initial initialization can take o(n) time.

in simple cases 1 can estimate benefit of parallelisation of algorithm, (as case cuda) static overhead of using parallel processing might take more time, parallel processing saves.

so, in opinion, simpler write application, measure it's performance , use plot amdahl's curve trying a priori correctly estimate nuances of algorithm , hardware. in case such estimations made, obvious without "laws".


Comments

Post a Comment

Popular posts from this blog

commonjs - How to write a typescript definition file for a node module that exports a function? -

consider, toml node module can use: // toml.d.ts declare module toml { export function parse(value:string):any; } declare module "toml" { export = toml; } then: /// <reference path="../../../../../defs/toml/toml.d.ts"/> import toml = require('toml'); toml.parse(...); however, node module export single function, such 'glob' ( https://github.com/isaacs/node-glob ). the node usage of module be: var glob = require("glob") glob("*.js", {}, callback(err, files) { ... }); you'd naively expect this: // glob.d.ts declare function globs(paths:string, options:any, callback:{(err:any, files:string[]):void; ...but because typescripts 'import' semantics bit strange, seems can use 'import .. = require()' statement alias modules . trying call: /// <reference path="../../../../../defs/glob/glob.d.ts"/> import blog = require('glob'); results in: error ts2072: mod...
Read more

openid - Okta: Failed to get authorization code through API call -

i'm integrating okta own idp server using okta's api. i'm implementing authorization code flow following steps below: in own server, use /api/v1/authn endpoint sessiontoken. use sessiontoken obtain authorization calling endpoint: /oauth2/v1/authorize?client_id=" + clientid + "&sessiontoken=" + sessiontoken + "&response_type=code&response_mode=query&scope=openid&redirect_uri=" + redirecturl + "&state=evanyang&nonce=" it's supposed return response status code 302 , location header containing redirect url code value. however, keep getting response status code 200 , without location header, html body saying "you using unsupported browser." , "javascript disabled on browser." according api documentation: http://developer.okta.com/docs/api/resources/oidc.html#authentication-request , sessiontoken parameter sufficient this: an okta one-time sessiontoken. allows api-based ...
Read more

ios - Change Storyboard View using Seague -

i have following code checks connection server, , depending on want change views if no connection found. nsstring *connect = [nsstring stringwithcontentsofurl:[nsurl urlwithstring:@"http://myserver.com"] encoding:nsutf8stringencoding error:nil]; if (connect == null) { [self performseguewithidentifier:@"network_failure" sender:self]; } else { // other code here } my problem not change views. have added storyboard segue between 2 views identifier of network_failure . not have navigation controller or that, problem? thanks help! no not problem. first of all, in objectivec exists nil object oriented null . must use that. so: if(connect == nil) or better: if(!connect) so due fact nil not same of null , probably code does't enter in if scope . you have been able discover breakpoint. anyway: be sure set trigger segue (line) 2 view controller , , name network_failure (that not best name. should instead view1toview2 ) as, m...
Read more