java - BufferedReader taking too long -

this question has answer here:

• java - best approach parse huge (extra large) json file 2 answers
• outofmemory exception in lot of memory

this read file faster not write it. have 150mb file has json object inside it. use following code read it:

string filename ="/tmp/filetoread"; bufferedreader reader = new bufferedreader(new inputstreamreader(new fileinputstream(filename), charset.forname("utf-8"))); decompressedstring = reader.readline(); reader.close(); jsonobject obj = new jsonobject(decompressedstring); jsonarray profiledata = obj.getjsonobject("profiledata").getjsonarray("children"); .... 

it single line file , since json can't split ( or atleast think so). reading file gives me outofmemory error or tle. file takes more 7 secs read , results in tle since execution of whole code cannot go beyond 7 seconds. oom on decompressedstring = reader.readline();.

is there way can reduce memory used or time takes read completely?

you have several problems @ hand:

1. you're preemptively parsing much.

   the error happens when read line since said "i oom on decompressedstring = reader.readline();".

   you should never try read data line line. bufferedreader.readline() block until you've read character \r or \n or sequence \r\n. when processing data of length, you're never sure you'll 1 of characters. also, you're never sure you'll of characters outside of data itself. string may long or malformed. don't ever pretend know format. bufferedreader.readline() must used when parsing, not when acquiring data.

2. you're not using appropriate library use-case

   reading json important, yes, you're reading @ once. when creating json, might want build stream (one of inputstream, reader or nio's channel/buffer).

   currently you're making json string. huge one. can safely assume you're going require @ 1 point twice memory need. 1 time in string , 1 time in finalized object.

   to reduce that, use appropriate library can pass 1 of stream mentioned above. mentioned in comments following: gson, json.simple , jackson.

3. your file may big anyways.

   if data , want acquire subset of (here, want under {"profiledata":{"children": <data>}}). have way much. how many elements exist @ same level profiledata? how many elements exist @ same level children? know? way much. not under profiledata.children useless. percentage of total data that? 50%? 90%? 99%?

   to solve this, want 1 of 2 things: want less data or want able focus request.

   if want less data, ask data provider give less: need. why more that? makes no sense. tell him , "i want less".

   if want focused data, use library allows both parse , reduce amount of data. might want have library lets this: "parse json , return processingdata.children element". unfortunately know no library it. if others do, please add comment or answer. apparently, gson able if use jsonreader , selectively use skipvalue().