Skip to main content
Visitor II
June 18, 2020
Solved

How can I to store variables into a csv log file?

  • June 18, 2020
  • 7 replies
  • 4594 views

I see file storage saves a json. I tried to put a csv conversion node in the middle but nothing is saved.

Thanks

    This topic has been closed for replies.
    Best answer by stephane.legargeant

    Hello

    The .stcm may not be processed properly by the node-red-contrib-json2csv because it is a list of json blocks. So each block should be parsed separately.

    Here is a proposal to convert the stcm files in csv. It will output on each line of the csv file the variable name, x and y.

    0693W000003PAbFQAW.png

    Here is the flow to import :

    [{"id":"cc0c8ae8.c69f98","type":"tab","label":"Convert stcm","disabled":false,"info":"This flow is used to convert stcm file in csv file\n\nThe file name must be entered in the \"set file name\" node\n\nThe csv output format is variable Name, x, y\nNote : the data may be processed by group, so the x value may not be in time order. The csv file may be sorted on the x value to ensure order coherency. "},{"id":"a9caceb2.eb94d","type":"file in","z":"cc0c8ae8.c69f98","name":"Read file","filename":"","format":"lines","chunk":false,"sendError":false,"encoding":"none","x":280,"y":360,"wires":[["99d027a1.b92d98"]]},{"id":"9350e2b8.9aa3e","type":"inject","z":"cc0c8ae8.c69f98","name":"Start","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":110,"y":160,"wires":[["3371f5e6.b3b50a"]]},{"id":"99d027a1.b92d98","type":"json","z":"cc0c8ae8.c69f98","name":"Parse each line","property":"payload","action":"","pretty":false,"x":480,"y":360,"wires":[["badf8efa.bacee"]]},{"id":"badf8efa.bacee","type":"function","z":"cc0c8ae8.c69f98","name":"Format data","func":"let output=\"\";\nmsg.payload.variabledata.forEach (data=>output=output+ msg.payload.variablename+ \",\"+ data.x + \",\"+ data.y+\"\\n\");\nmsg.payload= output;\nreturn msg;","outputs":1,"noerr":0,"x":690,"y":360,"wires":[["38fb106e.c00f"]]},{"id":"d2859af6.e93368","type":"file","z":"cc0c8ae8.c69f98","name":"Write csv","filename":"","appendNewline":false,"createDir":false,"overwriteFile":"false","encoding":"none","x":920,"y":580,"wires":[[]]},{"id":"3371f5e6.b3b50a","type":"change","z":"cc0c8ae8.c69f98","name":"Set file name","rules":[{"t":"set","p":"filename","pt":"msg","to":"C:\\GIT\\test\\test3.stcm","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":330,"y":160,"wires":[["a9caceb2.eb94d","593d0b11.8d3654"]]},{"id":"f3d3a57a.c56e48","type":"comment","z":"cc0c8ae8.c69f98","name":"Enter your filename here","info":"","x":330,"y":120,"wires":[]},{"id":"38fb106e.c00f","type":"change","z":"cc0c8ae8.c69f98","name":"Set csv extension in filename","rules":[{"t":"change","p":"filename","pt":"msg","from":"stcm","fromt":"str","to":"csv","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":700,"y":580,"wires":[["d2859af6.e93368"]]},{"id":"b1ba9fe.177b06","type":"file","z":"cc0c8ae8.c69f98","name":"Eraze csv","filename":"","appendNewline":true,"createDir":false,"overwriteFile":"true","encoding":"none","x":780,"y":160,"wires":[[]]},{"id":"593d0b11.8d3654","type":"change","z":"cc0c8ae8.c69f98","name":"Format csv header ","rules":[{"t":"change","p":"filename","pt":"msg","from":"stcm","fromt":"str","to":"csv","tot":"str"},{"t":"set","p":"payload","pt":"msg","to":"Variable, x, y","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":590,"y":160,"wires":[["b1ba9fe.177b06"]]},{"id":"796c6b18.2aab34","type":"comment","z":"cc0c8ae8.c69f98","name":"Process the stcm file","info":"","x":490,"y":320,"wires":[]},{"id":"c2ec88a.7368678","type":"comment","z":"cc0c8ae8.c69f98","name":"Write the csv file","info":"","x":820,"y":540,"wires":[]}]

    To use it, go to menu import , copy paste the text, and select "new flow".

    Then for each conversion :

    1) in the box "Set file name", enter the name of file to convert (absolute path + extension : "C:\log\test1.stcm")

    2) press start. The file will be processed and stored with the csv extension.

    I hope it will answer your need.

    Stephane

    7 replies

    Visitor II
    June 22, 2020

    Dear @Franco Spediacci​ ,

    Can you share your flow to try to understand your issue ?

    Landry

    Visitor II
    June 23, 2020

    Hi Landry,

    thanks for reply.

    You can find two flows attached and related logs.

    In one I do not use conversion. In the log I see json.

    In the other I use CSV conversion. In the log I see only the variables names.

    Franco

    Visitor II
    August 5, 2020

    I have the exact same issue, I can only see the variable names.

    Technical Moderator
    August 6, 2020

    Hello @Franco Spediacci​  and @david.rojas​ 

    You can try json-2-csv node to convert json to csv

    https://flows.nodered.org/node/node-red-contrib-json2csv

    Hope it helps

    Richard.

    Visitor II
    August 10, 2020

    Same issue here, did anyone find a way to export data to csv?

    ST Employee
    August 12, 2020

    Hello

    The .stcm may not be processed properly by the node-red-contrib-json2csv because it is a list of json blocks. So each block should be parsed separately.

    Here is a proposal to convert the stcm files in csv. It will output on each line of the csv file the variable name, x and y.

    0693W000003PAbFQAW.png

    Here is the flow to import :

    [{"id":"cc0c8ae8.c69f98","type":"tab","label":"Convert stcm","disabled":false,"info":"This flow is used to convert stcm file in csv file\n\nThe file name must be entered in the \"set file name\" node\n\nThe csv output format is variable Name, x, y\nNote : the data may be processed by group, so the x value may not be in time order. The csv file may be sorted on the x value to ensure order coherency. "},{"id":"a9caceb2.eb94d","type":"file in","z":"cc0c8ae8.c69f98","name":"Read file","filename":"","format":"lines","chunk":false,"sendError":false,"encoding":"none","x":280,"y":360,"wires":[["99d027a1.b92d98"]]},{"id":"9350e2b8.9aa3e","type":"inject","z":"cc0c8ae8.c69f98","name":"Start","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":110,"y":160,"wires":[["3371f5e6.b3b50a"]]},{"id":"99d027a1.b92d98","type":"json","z":"cc0c8ae8.c69f98","name":"Parse each line","property":"payload","action":"","pretty":false,"x":480,"y":360,"wires":[["badf8efa.bacee"]]},{"id":"badf8efa.bacee","type":"function","z":"cc0c8ae8.c69f98","name":"Format data","func":"let output=\"\";\nmsg.payload.variabledata.forEach (data=>output=output+ msg.payload.variablename+ \",\"+ data.x + \",\"+ data.y+\"\\n\");\nmsg.payload= output;\nreturn msg;","outputs":1,"noerr":0,"x":690,"y":360,"wires":[["38fb106e.c00f"]]},{"id":"d2859af6.e93368","type":"file","z":"cc0c8ae8.c69f98","name":"Write csv","filename":"","appendNewline":false,"createDir":false,"overwriteFile":"false","encoding":"none","x":920,"y":580,"wires":[[]]},{"id":"3371f5e6.b3b50a","type":"change","z":"cc0c8ae8.c69f98","name":"Set file name","rules":[{"t":"set","p":"filename","pt":"msg","to":"C:\\GIT\\test\\test3.stcm","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":330,"y":160,"wires":[["a9caceb2.eb94d","593d0b11.8d3654"]]},{"id":"f3d3a57a.c56e48","type":"comment","z":"cc0c8ae8.c69f98","name":"Enter your filename here","info":"","x":330,"y":120,"wires":[]},{"id":"38fb106e.c00f","type":"change","z":"cc0c8ae8.c69f98","name":"Set csv extension in filename","rules":[{"t":"change","p":"filename","pt":"msg","from":"stcm","fromt":"str","to":"csv","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":700,"y":580,"wires":[["d2859af6.e93368"]]},{"id":"b1ba9fe.177b06","type":"file","z":"cc0c8ae8.c69f98","name":"Eraze csv","filename":"","appendNewline":true,"createDir":false,"overwriteFile":"true","encoding":"none","x":780,"y":160,"wires":[[]]},{"id":"593d0b11.8d3654","type":"change","z":"cc0c8ae8.c69f98","name":"Format csv header ","rules":[{"t":"change","p":"filename","pt":"msg","from":"stcm","fromt":"str","to":"csv","tot":"str"},{"t":"set","p":"payload","pt":"msg","to":"Variable, x, y","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":590,"y":160,"wires":[["b1ba9fe.177b06"]]},{"id":"796c6b18.2aab34","type":"comment","z":"cc0c8ae8.c69f98","name":"Process the stcm file","info":"","x":490,"y":320,"wires":[]},{"id":"c2ec88a.7368678","type":"comment","z":"cc0c8ae8.c69f98","name":"Write the csv file","info":"","x":820,"y":540,"wires":[]}]

    To use it, go to menu import , copy paste the text, and select "new flow".

    Then for each conversion :

    1) in the box "Set file name", enter the name of file to convert (absolute path + extension : "C:\log\test1.stcm")

    2) press start. The file will be processed and stored with the csv extension.

    I hope it will answer your need.

    Stephane

    Visitor II
    August 12, 2020

    It works, thank you!

    It still would be nice to have this function directly inside STM32CubeMonitor.

    ST Employee
    March 25, 2021