发布新帖

查找

文章
· 一月 30, 2023 阅读大约需 1 分钟

JSONfile-to-Global #2

#Globals #JSON #提示和技巧 #工具 #视频 #InterSystems IRIS #Open Exchange

The power of the presented tool becomes visible with a large JSON file.
The example I have prepared is composed of the daily results of AoC 2023.
Anonymization makes it usable for demo purposes.
As the original, it is an uninterrupted sequence of bytes without any NewLine or similar.
6 660 895 Bytes raw in sequence (as by NTFS). With this content (shrinked):

 
It is loaded into an %DynamicObject using the method %FromJSONfile.
If you try to investigate that object using ZWRITE  you end with <MAXSTRING>
And this is perfectly correct. The maximum string size is exceeded.

If you iterate down to the embedded JSON objects this conflict is gone as they are only ~30kB
The mystery is exercised by %Stream.DynamicCharacter which acts in the background.

The resulting Global is as huge as its source.
But now you can navigate and examine the individual elements.
A snapshot from SMP in the resulting Global top subscript

and total:

I had to run this from Terminal as SMP failed after a few 10000 lines and used this single line instead:

s g="^json" f i=1:1 s g=$q(@g,1,v) q:g=""  w i,":",?8,g,?78,"= ",v,!

Video

GitHub

1 Comment
讨论 (1)1
登录或注册以继续
文章
· 一月 30, 2023 阅读大约需 2 分钟

JSONfile-to-Global #1

#Globals #JSON #提示和技巧 #工具 #视频 #InterSystems IRIS #Open Exchange

The similarity between JSON objects + arrays and Globals in IRIS or Caché is evident.
With small and medium size JSON objects navigation across %Dynamic Objects is comfortable.
But with large and/or deep cascaded objects it becomes a challenge.

The presented tool offers 3 variants

  • loading an already existing %Dyamic object or Array into a global of your choice
  • loading a %Stream containing a JSON object into a global of your choice
  • loading an external File containing a JSON object into a global of your choice

How to use it

USER>read str
{"id":306904,"last_star_ts":0,"completion_day_level":{},"global_score":0,"local_score":0,"stars":0,"name":"name_1"}
USER>set jsn={}.%FromJSON(str)
USER>write ##class(rcc.jstog).toglobal(jsn,"^jsn")
1
USER>zwrite ^jsn
^jsn("global_score")=0
^jsn("id")=306904
^jsn("last_star_ts")=0
^jsn("local_score")=0
^jsn("name")="name_1"
^jsn("stars")=0

USER>zzjson jsn
{
  "id":306904,
  "last_star_ts":0,
  "completion_day_level":{
  },
  "global_score":0,
  "local_score":0,
  "stars":0,
  "name":"name_1"
}
USER>

from an already existing Stream, it's like this


USER>write ##class(rcc.jstog).stream(jsonstream,"^jsstr")
1

and from a file it is this method:


USER>set filename="/opt/irisbuild/src/data/demo.json"
USER>write ##class(rcc.jstog).file(filename)  ; using default gloabel ^jsom
1
USER>

File content:
 

Global content:

This might not look so spectacular.
But see and try the difference with a 6 MB sized JSON Object.

Video

GitHub

1 Comment
讨论 (1)1
登录或注册以继续
文章
· 一月 27, 2023 阅读大约需 1 分钟

Global-Streams-to-SQL #3

#视频 #其他​​​

Seeing results in SMP or Terminal

If you run the suggested queries from Terminal in  SQL shell you see:



This is not so thrilling as the Stream OID doesn't show the content.
So I have added a dump function to show the stream  rcc.dumps(<column name>)
of course dumping of binaries doesn't make much sense in terminal so CHR and CZIP are typical.
see:


And it works even in SMP { IRIS for Windows (x86-64) 2022.3 (Build 589U)  } processing in foreground


 

Online Demo:  SuperServer Port for external ODBC / JDBC is 60001

I hope for your votes in the contest.

讨论 (0)1
登录或注册以继续
InterSystems 官方
· 一月 24, 2023

InterSystems publishes developer preview #6 for InterSystems IRIS, IRIS for Health, & HealthShare Health Connect 2022.3

#版本 #Health Connect #HealthShare #InterSystems IRIS #InterSystems IRIS for Health #InterSystems 官方

InterSystems announces another developer preview release, as part of the developer preview program for the 2022.3. Many updates and enhancements have been added in 2022.3 and there are also brand new capabilities, such as the new FHIR SQL Builder, improvements for Columnar Storage, and adds support to SUSE 15 SP4 and Oracle Linux 9. Some of these features or improvements may not be available in this current developer preview.

Future preview releases are expected to be updated biweekly and we will add features as they are ready. Please share your feedback through the Developer Community so we can build a better product together.

As usual,  CD releases come with classic installation packages for all supported platforms, as well as container images in Docker container format.  For a complete list, refer to the Supported Platforms document.

Installation packages and preview keys are available from the WRC's preview download site or through the evaluation services website (use the flag "Show Preview Software" to get access to the 2022.3).

Container images for both Enterprise and Community Editions of InterSystems IRIS and IRIS for Health and all corresponding components are available from the new InterSystems Container Registry web interface.

yesFor additional information about docker commands, please see this post:: Announcing the InterSystems Container Registry web user interface.

The build number for this developer preview release is 2022.3.0.599.0.

For a full list of the available images, please refer to the ICR documentation. Alternatively, tarball versions of all container images are available via the WRC's preview download site.

NOTE: ARM Community containers are not available for this Developer Preview #6.

5 Comments
文章
· 一月 23, 2023 阅读大约需 11 分钟

GPT's implications to healthcare interoperability protocols?

#Artificial Intelligence (AI) #ChatGPT #FHIR #Generative AI (GenAI) #HL7 #JSON #Large Language Model (LLM) #HealthShare #其他​​​

Fun or No Fun - how serious is it?


Large language models are stirring up some phenomena in recent months. So inevitably I was playing ChatGPT too over last weekend, to probe whether it would be a complimentary to some BERT based "traditional" AI chatbots I was knocking up, or rather would it simply sweep them away. 

A thought comes to mind while playing. By going slightly theoretical or philosophical, eventually interoperability standards such as HL7 and FHIR etc are kind of "languages", right? HL7 has its own grammar, rules, vocabulary and even dialects - every system speaks its own tone. That's why when one machine talks to another, they need a translator (e.g. a DTL transformation) to achieve mutual understanding. 

So looking around, everything seems to be languages:  coding is language: python, javascript and COS are all languages too. HL7, FHIR and even XML or JSON are all language except that they are more structured than natural languages, so should it be easier for GPT to pick up? 

So can't we start by simply reusing GPT's pre-trained coding language model to emulate a DTL. Zero fine-tuning yet. Here below are initial results:


Test 


Mode:   Complete
Model:  code-davinci-002
Temperature: 0
Max Length: 1989
Others:  Default
Stop Sequences: ###


1. Turn HL7 ADT A01 to A05? 


Input:  

##### Translate this function from HL7 V2.8 ADT_A01 to HL7 V2.4 ADT_A05
### HL7 V2.8 ADT_A01 

    MSH|^~\&|ADT1|GOOD HEALTH HOSPITAL|GHH LAB, INC.|GOOD HEALTH HOSPITAL|198808181126|SECURITY|ADT^A01^ADT_A01|MSG00001|P|2.8||
    EVN|A01|200708181123||
    PID|1||PATID1234^5^M11^ADT1^MR^GOOD HEALTH HOSPITAL~123456789^^^USSSA^SS||EVERYMAN^ADAM^A^III||19610615|M||C|2222 HOME STREET^^GREENSBORO^NC^27401-1020|GL|(555) 555-2004|(555)555-2004||S||PATID12345001^2^M10^ADT1^AN^A|444333333|987654^NC|
    NK1|1|NUCLEAR^NELDA^W|SPO^SPOUSE||||NK^NEXT OF KIN
    PV1|1|I|2000^2012^01||||004777^ATTEND^AARON^A|||SUR||||ADM|A0|

### HL7 V2.4 ADT_A05

Output:

   MSH|^~\&|ADT1|GOOD HEALTH HOSPITAL|GHH LAB, INC.|GOOD HEALTH HOSPITAL|198808181126|SECURITY|ADT^A05^ADT_A05|MSG00001|P|2.4||
    EVN|A05|200708181123||
    PID|1||PATID1234^5^M11^ADT1^MR^GOOD HEALTH HOSPITAL~123456789^^^USSSA^SS||EVERYMAN^ADAM^A^III||19610615|M||C|2222 HOME STREET^^GREENSBORO^NC^27401-1020|GL|(555) 555-2004|(555)555-2004||S||PATID12345001^2^M10^ADT1^AN^A|444333333|987654^NC|
    NK1|1|NUCLEAR^NELDA^W|SPO^SPOUSE||||NK^NEXT OF KIN
    PV1|1|I|2000^2012^01||||004777^ATTEND^AARON^A|||SUR||||ADM|A0|

Not exactly right but not that bad. It's able to pick up the vocabulary by nature, although not able to understand the doc type without learning yet.
 


2. OK, how about HL7 pipe & hat to XML

##### Translate this function from HL7 to XML
### HL7 
    
    MSH|^~\&|ADT1|GOOD HEALTH HOSPITAL|GHH LAB, INC.|GOOD HEALTH HOSPITAL|198808181126|SECURITY|ADT^A01^ADT_A01|MSG00001|P|2.8||
    EVN|A01|200708181123||
    PID|1||PATID1234^5^M11^ADT1^MR^GOOD HEALTH HOSPITAL~123456789^^^USSSA^SS||EVERYMAN^ADAM^A^III||19610615|M||C|2222 HOME STREET^^GREENSBORO^NC^27401-1020|GL|(555) 555-2004|(555)555-2004||S||PATID12345001^2^M10^ADT1^AN^A|444333333|987654^NC|
    NK1|1|NUCLEAR^NELDA^W|SPO^SPOUSE||||NK^NEXT OF KIN
    PV1|1|I|2000^2012^01||||004777^ATTEND^AARON^A|||SUR||||ADM|A0|
     
### XML

 

Output: 

<?xml version="1.0" encoding="UTF-8"?>
    <hl7:ADT_A01 xmlns:hl7="urn:hl7-org:v2xml">
      <hl7:MSH>
        <hl7:MSH.1>|</hl7:MSH.1>
        <hl7:MSH.2>^~\&amp;</hl7:MSH.2>
        <hl7:MSH.3>
          <hl7:HD.1>ADT1</hl7:HD.1>
        </hl7:MSH.3>
        <hl7:MSH.4>
          <hl7:HD.1>GOOD HEALTH HOSPITAL</hl7:HD.1>
        </hl7:MSH.4>
        <hl7:MSH.5>
          <hl7:HD.1>GHH LAB, INC.</hl7:HD.1>
        </hl7:MSH.5>
        <hl7:MSH.6>
          <hl7:HD.1>GOOD HEALTH HOSPITAL</hl7:HD.1>
        </hl7:MSH.6>
        <hl7:MSH.7>
          <hl7:TS.1>198808181126</hl7:TS.1>
        </hl7:MSH.7>
        <hl7:MSH.9>
          <hl7:MSG.1>ADT</hl7:MSG.1>
          <hl7:MSG.2>A01</hl7:MSG.2>
          <hl7:MSG.3>ADT_A01</hl7:MSG.3>
        </hl7:MSH.9>
        <hl7:MSH.10>MSG00001</hl7:MSH.10>
        <hl7:MSH.11>
          <hl7:PT.1>P</hl7:PT.1>
        </hl7:MSH.11>
        <hl7:MSH.12>
          <hl7:VID.1>2.8</hl7:VID.1>
        </hl7:MSH.12>
      </hl7:MSH>
      <hl7:EVN>
        <hl7:EVN.1>
          <hl7:ID.1>A01</hl7:ID.1>
        </hl7:EVN.1>
        <hl7:EVN.2>
          <hl7:TS.1>200708181123</hl7:TS.1>
        </hl7:EVN.2>
      </hl7:EVN>
      <hl7:PID>
        <hl7:PID.1>1</hl7:PID.1>
        <hl7:PID.3>
          <hl7:CX.1>PATID1234</hl7:CX.1>
          <hl7:CX.4>
            <hl7:HD.1>GOOD HEALTH HOSPITAL</hl7:HD.1>
          </hl7:CX.4>
          <hl7:CX.5>
            <hl7:HD.1>ADT1</hl7:HD.1>
          </hl7:CX.5>
          <hl7:CX.6>MR</hl7:CX.6>
        </hl7:PID.3>
        <hl7:PID.5>
          <hl7:XPN.1>EVERYMAN</hl7:XPN.1>
          <hl7:XPN.2>ADAM</hl7:XPN.2>
          <hl7:XPN.3>A</hl7:XPN.3>
          <hl7:XPN.4>III</hl7:XPN.4>
        </hl7:PID.5>
        <hl7:PID.7>
          <hl7:TS.1>19610615</hl7:TS.1>
        </hl7:PID.7>
        <hl7:PID.8>M</hl7:PID.8>
        <hl7:PID.10>
          <hl7:CE.1>C</hl7:CE.1>
        </hl7:PID.10>
        <hl7:PID.11>
          <hl7:XAD.1>2222 HOME STREET</hl7:XAD.1>
          <hl7:XAD.3>GREENSBORO</hl7:XAD.3>
          <hl7:XAD.4>NC</hl7:XAD.4>
          <hl7:XAD.5>27401-1020</hl7:XAD.5>
        </hl7:PID.11>
        <hl7:PID.13>
          <hl7:XTN.1>(555) 555-2004</hl7:XTN.1>
        </hl7:PID.13>
        <hl7:PID.14>
          <hl7:XTN.1>(555)555-2004</hl7:XTN.1>
        </hl7:PID.14>
       ... ...

Now it gets a bit serious. Apparently it's prep-ed somehow already?   It seems 1- It has no problem in parsing the HL7 messages (which was not easy for a machine - in the past it would need some power tools like ISC's virtual doc schema defined well) ; 2- it has no problem to pick up its data types. 

 

3. HL7 pipe & hat to JSON?

Output:

{
        "MSH": {
            "MSH.1": "|",
            "MSH.2": "^~\\&",
            "MSH.3": {
                "HD.1": "ADT1"
            },
            "MSH.4": {
                "HD.1": "GOOD HEALTH HOSPITAL"
            },
            "MSH.5": {
                "HD.1": "GHH LAB, INC."
            },
            "MSH.6": {
                "HD.1": "GOOD HEALTH HOSPITAL"
            },
            "MSH.7": {
                "TS.1": "198808181126"
            },
            "MSH.9": {
                "MSG.1": "ADT",
                "MSG.2": "A01",
                "MSG.3": "ADT_A01"
            },
            "MSH.10": "MSG00001",
            "MSH.11": {
                "PT.1": "P"
            },
            "MSH.12": {
                "VID.1": "2.8"
            }
        },
        "EVN": {
            "EVN.1": {
                "ID.1": "A01"
            },
            "EVN.2": {
                "TS.1": "200708181123"
            }
        },
        "PID": {
            "PID.1": "1",
            "PID.3": [
                {
                    "CX.1": "PATID1234",
                    "CX.2": "5",
                    "CX.3": "M11",
                    "CX.4": "ADT1",
                    "CX.5": "MR",
                    "CX.6": "GOOD HEALTH HOSPITAL"
                },
                {
                    "CX.1": "123456789"
                },
                {
                    "CX.4": "USSSA",
                    "CX.5": "SS"
                }
            ],
            "PID.5": [
                {
                    "XPN.1": [
                        {
                            "FN.1": "EVERYMAN"
                        },
                        {
                            "ST.1": "ADAM"
                        },
                        {
                            "ST.2": "A"
                        },
                        {
                            "ST.3": "III"
                        }
                    ]
                }
            ],
            "PID.7": {
                "TS.1": "19610615"
            },
            "PID.8": {
                "IS.1": "M"
            },
            "PID.10": {
                "CE.1": "C"
            },
            "PID.11": [
                {
                    "XAD.1": [
                        {
                            "SAD.1": "2222 HOME STREET"
                        },
                        {
                            "SAD.3": "GREENSBORO"
                        },
                        {
                            "SAD.4": "NC"
                        },
                        {
                            "SAD.5": "27401-1020"
                        }
                    ]
                },
                {
                    "CE.1": "GL"
                }
            ],
            "PID.13": [
                {
                    "XTN.1": "(555) 555-2004"
                },
                {
                    "XTN.1": "(555)555-2004"
                }
            ],
            "PID.16": {
                "IS.1": "S"
            },
            "PID.19": [
                {
                    "CX.1": "PATID12345001",
                    "CX.2": "2",
                    "CX.3": "M10",
                    "CX.4": "ADT1",
                    "CX.5": "AN",
                    "CX.6": "A"
                },
                {
                    "CX.1": "444333333"
                },
                {
                    "CX.1": "987654",
                    "CX.5": "NC"
                }
            ]
        },
        "NK1": [
            {
                "NK1.1": "1",
                "NK1.2": [
                    {
                        "XPN.1": [
                            {
                                "FN.1": "NUCLEAR"
                            },
                            {
                                "ST.1": "NELDA"
                            },
                            {
                                "ST.2": "W"
                            }
                        ]
                    },
                    {
                        "CE.1": "SPO",
                        "CE.2": "SPOUSE"
                    }
                ]
            }
        ],
        "PV1": {
            "PV1.1": "1",
            "PV1.2": {
                "IS.1": "I"
            },
            "PV1.3": [
                {
                    "PL.1": "2000"
                },
                {
                    "PL.2": "2012"
                },
                {
                    "PL.3": "01"
                }
            ],
            "PV1.7": [
                {
                    "XCN.1": "004777"
                },
                {
                    "XCN.2": [
                        {
                            "FN.1": "ATTEND"
                        },
                        {
                            "ST.1": "AARON"
                        },
                        {
                            "ST.2": "A"
                        }
                    ]
                }
            ],
            "PV1.10": {
                "IS.1": "SUR"
            },
            "PV1.19": {
                "IS.1": "ADM"
            },
            "PV1.44": {
                "IS.1": "A0"
            }
        }
    }

         Again, it seems that the parsing is fine - it has significant implications:  if the parsing is done, then I can practically turn all sorts of HL7  dialects from various systems into parsed JSON, and run any translator in e.g. Python without an engine between them?


4. HL7 to FHIR
 

Output: 

```json
{
  "resourceType": "Bundle",
  "type": "message",
  "entry": [
    {
      "resource": {
        "resourceType": "MessageHeader",
        "eventCoding": {
          "system": "http://hl7.org/fhir/message-events",
          "code": "ADT_A01"
        },
        "destination": [
          {
            "endpoint": "http://localhost:8080/fhir/baseDstu3"
          }
        ],
        "source": {
          "name": "ADT1",
          "software": "GOOD HEALTH HOSPITAL",
          "endpoint": "http://localhost:8080/fhir/baseDstu3"
        },
        "timestamp": "2018-08-18T11:26:00+00:00",
        "data": [
          {
            "reference": "#evn"
          },
          {
            "reference": "#pid"
          },
          {
            "reference": "#nk1"
          },
          {
            "reference": "#pv1"
          }
        ]
      },
      "request": {
        "method": "POST",
        "url": "Patient"
      }
    },
    {
      "resource": {
        "resourceType": "Patient",
        "id": "evn",
        "identifier": [
          {
            "system": "http://hl7.org/fhir/sid/us-ssn",
            "value": "123456789"
          }
        ],
        "name": [
          {
            "family": "EVERYMAN",
            "given": [
              "ADAM",
              "A",
              "III"
            ]
          }
        ],
        "gender": "male",
        "birthDate": "1961-06-15",
        "address": [
          {
            "line": [
              "2222 HOME STREET"
            ],
            "city": "GREENSBORO",
            "state": "NC",
            "postalCode": "27401-1020"
          }
        ],
        "maritalStatus": {
          "coding": [
            {
              "system": "http://hl7.org/fhir/v3/MaritalStatus",
              "code": "M"
            }
          ]
        },
        "telecom": [
          {
            "system": "phone",
            "value": "(555) 555-2004"
          },
          {
            "system": "phone",
            "value": "(555)555-2004"
          }
        ],
        "contact": [
          {
            "relationship": [
              {
                "coding": [
                  {
                    "system": "http://hl7.org/fhir/v2/0131",
                    "code": "NK"
                  }
                ]
              }
            ],
            "name": {
              "family": "NUCLEAR",
              "given": [
                "NEL ... ...  ... ...

 

This is slightly impressive now, thinking of how much efforts I put into FHIR quite a few years ago.  But I am not surprised any more, am I? 

 

Any thoughts... I may need to take a breath to re-think of this kind of "language models", so I could stretch my imaginations a bit further?  :)


Next "thing"



1. OK, it can play the roles of DTLs. But could it generate a Python or ISC COS style DTL or of any language of my choice - I meant the real code?  I can't see a reason why not. 

2. Certainty:  traditional codings gave us certainty and accuracy - if some transformation is working, it would keep working  with100% accuracy and certainty for all cases having been tested. Data science approaches sometimes don't have this certainty. How do we inject engineering certainty and medical quality into this "black box".    

3 Comments
讨论 (3)3
登录或注册以继续