查找

问题
· 七月 10, 2023

Configure SFTP Operation to use Multi Factor Authentication

#身份认证 #业务运营 #FTP #Health Connect #InterSystems IRIS #InterSystems IRIS for Health

Hello,

I have a EnsLib.HL7.Operation.FTPOperation that uses SFTP protocol and public/private key to connect to an external vendor moveitcloud.

Issue: The vendor is planning to enable Multi Factor Authentication for this file transfer account.

Question: Have you configured a SFTP operation to use Multi factor Authentication? If not, is there another way?

Thank you,

1 Comment
讨论 (1)2
登录或注册以继续
文章
· 七月 10, 2023 阅读大约需 6 分钟

InterSystems Reports. Formulas, parameters, labels

#自适应分析（Adaptive Analytics） #InterSystems Reports (Logi) #Embedded Git

In this article, we will explore the use of parameters, formulas and labels in Logi Report Designer (formerly Logi JReport Designer). What are they for and how to create them?

Using the basic functionality of InterSystems Reports Designer, parameters, formulas and labels, you can significantly improve the detail and information content of the generated report. In addition, these tools allow you to automate some of the processes, which greatly speeds up and facilitates the creation of reports.

Let's analyze each tool separately now.

Parameters are individually configurable variables. They can store both static and dynamic data. For a dynamic parameter, you can set your own SQL query, which will be independent of the SQL query of the main dataset. In this way, you can output data without creating multiple datasets, thus keeping your design organized and clean. Parameters can be used both as a part of functions and SQL queries and as an independent indicator with the following syntax: “@YourParameter”. This quality is simply indispensable when the accuracy and detail of data are required.

A static parameter is a predefined value or a list of values ​​used as a condition or filter on the output. A static parameter can also be used in SQL queries, functions and labels using the same syntax: “@YourParameter”.

Formulas are fully programmable functions in the Java language. This powerful tool greatly expands the possibilities of analytics, allowing you to perform complex calculations and set logical conditions for the output data. Formulas are created in the built-in Java IDE inside Logi with a built-in set of functions for working with such data types as Array, Date / Time, Financial, Math, String, etc. Formulas can work with all data available in InterSystems Reports. The built-in IDE understands parameters, calculated fields, and even other formulas.

A label is a text widget. It serves to display any lowercase characters and variables. Its purpose speaks for itself: it is used in headers and footers, as a custom legend for a chart or table, in a word, and wherever data need to be titled. Just like all other InterSystems Reports tools, the label is a very flexible widget. It can be placed anywhere on the page, inside a graph or table.

Here we will look at examples of the most basic use of these tools.

We will create a parameter that returns the number of developers out of the total number of members of the InterSystems community.

In order to create a parameter, you need to click on the Catalog Manager button in the upper left corner of the toolbar.

In the window that opens, on the left, select the Parameters item, then click on the “New Parameter” button in the upper left corner.

The parameter creation window will open.

In the first line, we need to set the parameter name. It is recommended to choose a name that reflects the purpose of the parameter as accurately as possible and at the same time is short enough. It is necessary because during the development process, you will create an abundance of different parameters, and there is a risk of overloading the lists of parameters and functions. In this article, we will analyze the dynamic parameter, so in the second line in the Value Setting we will select Bind with Single Column. In the next Data Source line, we will choose the table from which the selection will occur. In our case, it is Members. Then in the Bind Column, we will select the columns from which we will return the value.

There isn't a separate column that can count the number of developers for us in the Members table. Yet, thanks to the ability to set a special SQL query, we can establish a condition for this particular selection. To do that, scroll down the list of properties, find the line Import SQL and click on it.

A request creation window will appear. It has already pre-recorded the selection string of the members_count column - the total number of participants. We only need to add the condition “where Developer = 1”. We can check the request by clicking on the Check button, and if it is successful, you should click OK.

After completing the previous steps, click OK in the parameter creation window, and the new Developer_member parameter will appear in the directory on the left. At this point, close the Catalog Manager window and try the newly created parameter. To do that, drag it to the report page and click View. After completing this step, you will be prompted to accept the value of the parameter taken from the current database, so click OK.

Ready! Now we can see how many people in the community are developers. This setting will automatically get updated every time the data changes.

Now we will create a formula. This formula will calculate the percentage of developers compared to the total number of participants.

To do that, you should repeat the same steps we took when creating the parameter. Let me remind you. Click on the Catalog Manager, select Formulas, and then New Formula in the upper left corner.

Before the formula creation window appears, we will be prompted to enter a name for the formula.

After that, the IDE will open for you to write code in Java. Built in InterSystems Reports, the compiler allows you to write short functions without defining classes and methods. In order to calculate the percentage, we need to divide our new parameter by the total number of members and multiply by 100. We have already created the CountTotalMembers indicator. We did it with the tool called InterSystems Adaptive Analytics (powered by AtScale). You can find out more about it here (link).

Thus, we got the following formula. Pay attention to the function allowing you to write comments to the code. Well-documented code will make it easier for others to work with your formulae.

After writing, you can check if the resulting code has any errors by clicking the checkmark button on the toolbar. Toolwill check whether the syntax is correct and whether the specified parameters are present.

After the function has been tested, it can be saved. To make that happen, click the Save button, and the formula will appear in the catalog.

Following the parameter example, our next step will be to drag the new formula onto the page and find out the share of developers in the total number of participants.

This function can be utilized as an indicator for graphs and tables. I will give you an example of using our new formula in a bar graph. You can learn more about charts and how to create them here (link).

In this example, I used a new formula and derived the proportion of developers over the past five months.

It's a bar graph, with the formula developers_to_members on the y-axis and the dimension month_year_num on the x-axis. It turned out a very visual trend and we did it in just a few minutes.

Now it is time to look at the labels.

They are embedded into the report page from the Insert tab with the Label button.

A window to enter the text will appear on the page.

The built-in functionality allows you to edit many parameters for the new Label. In this article, we used this widget to enhance our new table.

There are a lot of parameters placed on the panel on the right that can help you design a label. To make them appear, select the Label widget.

To set the borders for the widget, scroll down to the Border category. Select all 4 borders: Bottom Line, Left Line, Right Line, and Top Line, and set all of them to Solid.

In order to fill the background, you need to scroll through the properties to the Color category and select the desired shade. The same can be done in the Format tab on the toolbar at the top.

If you wish to select the font size, stay on the same Format tab, click the drop-down menu with sizes, and pick the desired one. It can also set the font type and the location of the text inside the widget.

For more precise positioning relative to other widgets, use the coordinates the Label has inside the page. By default, sheet sizes are in inches. The positioning settings are located in the Geometry category on the widget's properties panel on the left.

In this article, we tried to cover three basic features of InterSystems Reports (powered by Logi Report). That is why by now, we expect you to know how to create formulae, parameters, and labels with great confidence.

2 Comments
讨论 (2)3
登录或注册以继续
文章
· 七月 8, 2023 阅读大约需 2 分钟
Open Exchange

Character-Slice Index

#索引 #InterSystems IRIS

A recent question from @Vivian Lee reminded me of a rather ancient example.
It was the time when DeepSee's first version was released.
We got Bitmap Index.
And we got BitSlice Index: mapping a numeric value by its binary parts.
So my idea: Why not indexing strings by their characters?
The result of this idea was presented first in June 2008. 
IKnow wasn't publicly available at that time.

The principal idea was to split Strings into its characters
Data type %Text had some kind of capability in this direction.
But it is designed to split a text string into words.
The split is language dependent and requires a dictionary.

So I had to build my own dictionary. And borrowed it.
In Japanese every single character can be a word.
This was my start and with a few adjustments it serviced my needs 

The example that is also available now on IRIS and in Docker
consists of a Dictionary class and a Data class for demo data.

The result is impressive even with 3 documents of 158 lines
I compared normal [ (contains operator) to the character slices.

Search for a 2-char chunks: Global Access down from 159 to 34
Search for 2 independent chunks:  Global Access down from 159 to 15

The Dictionary defines what chunks are indexed.
This example uses chunks from 1..4 char.
The larger the chunks the larger the index and the better the hit rate.
It might be worth experimenting with your special cases. 

Github

 #Video   

#Video
 

1 Comment
讨论 (1)1
登录或注册以继续
讨论
· 七月 7, 2023

SAX XML Parser Error during SOAP request serialization (ERROR #6301: SAX XML Parser Error: <MAXSTRING>xeWrite+7)

#SOAP #InterSystems IRIS for Health

Hi everyone,

This one had me breaking a sweat 😅

When a SOAP request has a %Stream.GlobalBinary property, it fails to serialize to base64 when stream is larger than ~43MB with 

ERROR #6301: SAX XML Parser Error: <MAXSTRING>xeWrite+7^%occXMLInternal

Unless you enable the following parameters :  

Parameter BASE64LINEBREAKS = 1;
Parameter USEPPGHANDLER = 1;

The first one instructs %SOAP.WebBase to insert line breaks in the XML text, and the second one to use globals rather than local array memory during serialization (to avoid <STORE> error, depending on your available $zstorage)

Enabling both in my SOAP client (created by the SOAP Wizard), solved my problem : the IRIS business operation using the client can now process requests holding very large streams (in my particular case, a whopping 221MB !)

讨论 (0)1
登录或注册以继续
文章
· 七月 7, 2023 阅读大约需 6 分钟
Open Exchange

Using LLMs without Burning Dollars - Different Database Query Strategies

#API #Artificial Intelligence (AI) #ChatGPT #FHIR #Large Language Model (LLM) #InterSystems IRIS #Open Exchange

 

The continuing convergence of AI technologies and healthcare systems has brought forward many compelling advancements. Let's set the scene. If you have interacted with dynamic models like ChatGPT, you might have, like many of us, begun to envision its application using your unique datasets. Suppose in the healthcare sector you wish to link this technology with Electronic Health Records (EHR) or Electronic Medical Records (EMR), or perhaps you aim for heightened interoperability using FHIR's resources.  It all boils down to how we transfer/receive contextual-data to/from LLMs available in the market.

More accurate techniques include fine-tuning, training LLMs exclusively with the context datasets. Except, it costs millions of dollars to accomplish this today. The other way is to feed context to LLMs via one-shot or few-shot queries and getting an answer. Some ways in which this can be achieved are - generating SQL queries, generating code to query/parse, making calls with information from API specifications and so on. But, there is a problem of high token consumption and some of these answers may not be accurate always.

There’s no one solution fits all magic here, but understanding the pros and cons of these techniques can be helpful in devising your own strategy. Also, leveraging good engineering practices (like caches, secondary storage) and focussing on problem solving can help find a balance between the available methods. This post is an attempt at sharing some strategies and drawing comparison between them under different metrics.

Generating SQL queries

Firstly, we have the more conventional method - loading and parsing the SQL database structure and sample content through LangChain and executing GPT queries. This method has a track record of facilitating efficient and dynamic communication with our healthcare systems, marking itself as a tried-and-true technique in our industry.

There are solutions that pass just the database structure (table schema for example) and others that pass some redacted data to help the LLM generate accurate queries. The former solution has the advantage of fixed token usage and predictable costs but takes a hit on accuracy due to not being completely context-aware. The latter solution might be more token intensive and needs special care with anonymization techniques.  These solutions might be perfect for some use-cases, but could there exist a more optimal strategy?

Using LLMs to Generate code to navigate APIs and Database queries

Another sophisticated technique is to let the LLMs generate code to break down a question into multiple queries or API calls. This is a very natural way of solving complicated questions and unleashes the power of combining natural-language and underlying code.

This solution requires good prompt engineering and fine-tuning the template prompts to work well for all corner cases. Fitting this solution into an enterprise context can be challenging with the uncertainties in token usage, secure code generation and controlling the boundaries of what is and is not accessible by the generated code. But in it’s entirety the power of this technique to act autonomously to solve complex problems is fascinating and further advances in this area are something to look forward to.

Loading OpenAPI specs as context to LLMs

Our team wanted to try a different approach to control token usage but also leverage available context to get accurate results. How about employing LangChain to load and parse FHIR’s OpenAPI specifications? OpenAPI presents itself as an impactful alternative, furnished with adaptive and standardized procedures, validating the importance of FHIR's comprehensive API standards. Its distinct advantage lies in promoting effortless data exchange between diverse systems. The control here lies in being able to modify the specifications itself and not the prompts or generated outputs from the LLM.

Imagine the scenario: a POST API performs all required validating checks before data is added to the database. Now, envision leveraging that same POST API, but using a natural language method. It still carries out the same rigorous checks, ensuring consistency and reliability. This nature of OpenAPI doesn't just simplify interactions with healthcare services and applications, but also enhances API comprehensibility, making them easy to understand and predictable.

We understand this solution doesn’t hold the same power as autonomously breaking down tasks or generating code, but this is an aim at arriving at a more practical solution that can be adapted for most use-cases quickly.

Comparison

While all these techniques demonstrate unique benefits and the potential to serve different purposes, let us evaluate their performance against some metrics.

1. Reliability - Prioritizing reliability considering our alliance with AI, OpenAPI has an edge due to its utilization of standardized APIs. This ensures restricted unauthorized access and precise user authentication to specific data, providing enhanced data security as compared to passing AI-generated SQL for DB access - a method that could potentially raise reliability concerns.

2. Cost - The efficiency of the API’s filtering capabilities defined by FHIR plays a role in cost reduction. This permits only necessary data, streamlined through intense prompt engineering, to be transacted, unlike traditional DBs that may return more records than needed, leading to unnecessary cost surges.

3. Performance - The structured, and standardized presentation of data by OpenAPI specifications often contribute to superior output results from GPT-4 models, enhancing performance. However, SQL DBs can return results more swiftly for direct queries. It is important to account for Open API's potential for over-informing due to the definition of more parameters than might be needed for a query.

4. Interoperability - OpenAPI specifications shine when it comes to interoperability. Being platform-independent, they align perfectly with FHIR's mission to boost interoperability in healthcare, fostering a collaborative environment for seamless synchronization with other systems.

5. Implementation & Maintenance - Although it may be comparatively easier to spin off a DB and provide the context to the AI for querying makes the SQL database loading method with its lean control layer may seem easier to implement, the OpenAPI specifications, once mastered, offer benefits like standardization and easier maintenance that outweigh the initial learning and execution curve.

6. Scalability and Flexibility - SQL databases demand a rigid schema that may not comfortably allow for scalability and flexibility. Unlike SQL, OpenAPI offers a more adaptive and scalable solution, making it a future-friendly alternative.

7. Ethics and Concerns - An important, yet complex factor to consider given the rapid growth of AI. Would you be comfortable providing direct DB access to customers, even with filters and auth? Reflect on the importance of data de-identifiers in ensuring privacy within the healthcare space. Even though both OpenAPI and SQL databases have mechanisms to address these concerns, the inherent standardization provided by OpenAPI adds an additional layer of security.

While this discussion offers insights into some of the key factors to consider, it's essential to recognize that the choice between SQL, code generation and OpenAPI is multifaceted and subject to the specific requirements of your projects and organizations.

Please feel free to share your thoughts and perspectives on this topic - perhaps you have additional points to suggest or you'd like to share some examples that have worked best for your use-case.

Vote for our app in the Grand Prix contest if you find it promising!

1 Comment
讨论 (1)2
登录或注册以继续