c# - Newtonsoft Json deserializer not releasing memory -
i'm using streamreader jsontextreader deserialize large json file containing tens of thousands of small objects, , consuming way more memory think reasonable (and running out). i'm using understand recommended pattern reading large files.
code simplified expository purposes:
using (streamreader streamreader = new streamreader(stream)) using (jsontextreader reader = new jsontextreader(streamreader)) { jtoken token; while (reader.read() && reader.tokentype != jsontoken.endarray) { token = jtoken.load(reader); rawresult result = token.toobject<rawresult>(); results.add(result); } } the vs2015 memory profiler telling me of memory being consumed newtonsoft.json.linq.jvalue objects, bizarre because once current token has been converted toobject() there no reason (as far concerned) why shouldn't discarded.
i'm assuming newtonsoft library retaining of json parsed far in memory. don't need , think if prevent memory problems go away.
what can done?
it doesn't need using jtokens intermediary; deserialize directly rawresult class inside loop.
using (streamreader streamreader = new streamreader(stream)) using (jsontextreader reader = new jsontextreader(streamreader)) { var serializer = new jsonserializer(); while (reader.read() && reader.tokentype != jsontoken.endarray) { rawresult result = serializer.deserialize<rawresult>(reader); results.add(result); } } also note adding result items list, keeping them in memory. if can process them 1 @ time , write each result individually output (file, database, network stream, etc.) can save memory way also.
rawresult result = serializer.deserialize<rawresult>(reader); processresult(result); // process result instead of adding list
Comments
Post a Comment