MQTT picoC JsonParser

[Alex666]

Hello to all, I have a project where Loxone comunicate via MQTT and I need to parse Json data. Logical way for me was write a picoC code and it does work. But it looks like picoC in loxone do have issues with memory alocation (memory leak?) as after around about 7 minutes is stops because of "out of memory" error.

Have any of you had simmilar problem?

That's the test code..


// json keys names
enum Keys {
SUCCESS, PRICE, MAXSOC, MINSOC, STRICT, EVLIMIT, EXLIMIT, INLIMIT,
OPERATE, USEGRID, EVUSEBAT, USELIMIT, EVUSEGRID, PRICETIME, TARGETSOC,
CHTHRESHOLD, DISTOLIMIT, FORCEBATCH, INLIMITLOW, LIMITPRICE, FORCEBATDIS,
METERCUTOFF, PRICEAREAID, PRICETYPEID, TARGETPOWER, CURRENTPRICE,
TARGETSOCLOW, DISCTHRESHOLD, LOOP, NUM_KEYS
};

// json keys
char* keys[] = {
"success","price","maxSoc","minSoc","strict"," evLi mit","exLimit","inLimit",
"operate","useGrid","evusebat","useLimit","evu segr id","priceTime","targetSoc",
"chtreshold","disToLimit","forceBatCh","inLimi tLow ","limitPrice","forceBatDis",
"meterCutOff","priceareaid","pricetypeid","tar getP ower","currentPrice",
"targetSocLow","dischtreshold","loop"
};

while(TRUE)
{
char buf[64];
float values[29];
char* json = getinputtext(0); // T1 = JSON SETTINGS input
char* jsonReport getinputtext(1); //T2 JSON REPORT input

if(getinput(0) == 1)
{
setoutputtext(0, jsonReport); //only post report when input 1 ==1 (each 5s)
}

if(json != NULL)
{

// parser JSON-a
for(int k=0; k<NUM_KEYS; k++)
{
char search[32];
sprintf(search, ""%s":", keys[k]);

char* p = strstrskip(json, search);
if(p != NULL)
{
// skip spaces and other balasts "
while(*p && (*p == ' ' || *p == '"')) p++;

int i = 0;
// search till , } or "
while(*p && *p != ',' && *p != '}' && *p != '"' && i < sizeof(buf)-1)
buf[i++] = *p++;
buf[i] = 0;

values[k] = batof(buf); // put result into values
}
}

// drop some values to outputs
setoutput(0, values[PRICE]); // O0 = price
setoutput(1, values[MAXSOC]); // O1 = maxSoc
setoutput(2, values[MINSOC]); // O2 = minSoc
setoutput(3, values[STRICT]); // O3 = strict
setoutput(4, values[EVLIMIT]); // O4 = evLimit
setoutput(5, values[EXLIMIT]); // O5 = exLimit
setoutput(6, values[INLIMIT]); // O6 = inLimit
setoutput(7, values[OPERATE]); // O7 = operate
setoutput(8, values[USEGRID]); // O8 = useGrid
setoutput(9, values[EVUSEBAT]); // O9 = evusebat
setoutput(10, values[PRICETIME]); // O10 = PRICETIME
setoutput(11, values[LOOP]); // O11 = loop
}

sleeps(1); //sleep 1s as no need to work faster...
}

Best regards, Alex

[Noschvie]

It is certainly easier to leave the parsing of a JSON structure to a Loxberry.

[Alex666]

If that would be an option I would use it but it is not. I need working parser for a lot of existing projects that will be upgraded from rest api to MQTT and code do work just seems picoC in loxone do not handle memory well and there is no free() funcion supported for me to release memory after usage.

[Alex666]

Have to correct myself.. free() exists and that was the problem..

so if some need here it is functional jsonParse program...adjust names, buffers as per your need.


char buf[512];

float values[29]; //

char* json; // T1 JSON pointer to received MSG

char search[32];

char* p;




// json keys names

enum Keys {

SUCCESS, PRICE, MAXSOC, MINSOC, STRICT, EVLIMIT, EXLIMIT, INLIMIT,

OPERATE, USEGRID, EVUSEBAT, USELIMIT, EVUSEGRID, PRICETIME, TARGETSOC,

CHTHRESHOLD, DISTOLIMIT, FORCEBATCH, INLIMITLOW, LIMITPRICE, FORCEBATDIS,

METERCUTOFF, PRICEAREAID, PRICETYPEID, TARGETPOWER, CURRENTPRICE,

TARGETSOCLOW, DISCTHRESHOLD, LOOP, NUM_KEYS

};




// json keys

char* keys[] = {

"success","price","maxSoc","minSoc","strict"," evLi mit","exLimit","inLimit",

"operate","useGrid","evusebat","useLimit","evu segr id","priceTime","targetSoc",

"chtreshold","disToLimit","forceBatCh","inLimi tLow ","limitPrice","forceBatDis",

"meterCutOff","priceareaid","pricetypeid","tar getP ower","currentPrice",

"targetSocLow","dischtreshold","loop"

};




while(TRUE)

{

json = getinputtext(0); // T1 = JSON SETTINGS input




if(json != NULL)

{




// parser JSON-a

for(int k=0; k<NUM_KEYS; k++)

{

sprintf(search, ""%s":", keys[k]);

p = strstrskip(json, search);

if(p != NULL)

{

// skip spaces and other balasts "

while(*p && (*p == ' ' || *p == '"')) p++;




int i = 0;

// search till , } or "

while(*p && *p != ',' && *p != '}' && *p != '"' && i < sizeof(buf)-1)

buf[i++] = *p++;

buf[i] = 0;




values[k] = batof(buf); // put result into values

}

}




// drop some values to outputs

setoutput(0, values[PRICE]); // O0 = price

setoutput(1, values[MAXSOC]); // O1 = maxSoc

setoutput(2, values[MINSOC]); // O2 = minSoc

setoutput(3, values[STRICT]); // O3 = strict

setoutput(4, values[EVLIMIT]); // O4 = evLimit

setoutput(5, values[EXLIMIT]); // O5 = exLimit

setoutput(6, values[INLIMIT]); // O6 = inLimit

setoutput(7, values[OPERATE]); // O7 = operate

setoutput(8, values[USEGRID]); // O8 = useGrid

setoutput(9, values[EVUSEBAT]); // O9 = evusebat

setoutput(10, values[PRICETIME]); // O10 = PRICETIME

setoutput(11, values[LOOP]); // O11 = loop

free(p);

free(json);

}




sleeps(1); //sleep 1s as no need to work faster...

}

[Jan W.]

I have some doubt that the two free statements are required. According to the documentation there is no need to have a free statement for that command. Maybe the documentation is missing that? There are explicit hints to other commands that a free statement is required later.

"p" is just a pointer to an existing object and "strstrskip" does not allocate any memory, so there is no need to free any memory at the end.

However I do agree, that the declaration of search and other variables should be moved to the top as Pico C is executed by an interpreter.

Just my two cents regarding efficiency: the copy operation (sprintf(search, ""%s":", keys[k]) for each k does not look to be efficient in a loop to just to add the quotes and a colon each time. I would suggest to either move the operation away from the while-loop (of course you would need an array for "search", so a bit more of memory) or add the quotes and colon to the keys directly.