查找

文章
· 三月 4, 2024 阅读大约需 4 分钟

IKO - Lessons Learned (Part 2 - The IrisCluster)

We now get to make use of the IKO.

Below we define the environment we will be creating via a Custom Resource Definition (CRD). It lets us define something outside the realm of what the Kubernetes standard knows (this is objects such as your pods, services, persistent volumes (and claims), configmaps, secrets, and lots more). We are building a new kind of object, an IrisCluster object.

apiVersion: intersystems.com/v1alpha1
kind: IrisCluster
metadata:
  name: simple
spec:
  licenseKeySecret:
    #; to activate ISC license key
    name: iris-key-secret
  configSource:
    #; contains CSP-merge.ini, which is merged into IKO's
    #; auto-generated configuration.
    name: iris-cpf
  imagePullSecrets:
    - name: intersystems-pull-secret

  topology:
    data:
      image: containers.intersystems.com/intersystems/irishealth:2023.3
      compatibilityVersion: "2023.3"
    webgateway:
      replicas: 1
      image: containers.intersystems.com/intersystems/webgateway:2023.3
      applicationPaths:
        #; All of the IRIS instance's system default applications.
        #; For Management Portal only, just use '/csp/sys'.
        #; To support other applications, please add them to this list.
        - /csp/sys
        - /csp/broker
        - /api
        - /isc
        - /oauth2
        - /ui
        - /csp/healthshare
      alternativeServers: LoadBalancing
      loginSecret:
        name: iris-webgateway-secret

  serviceTemplate:
    # ; to enable external IP addresses
    spec:
      type: LoadBalancer

The IrisCluster object oversees and facilitates the deployment of all the components of our IRIS environment. In this specific environment we will have:

  • 1 IRIS For Health Instance (in the form of a data node)
  • 1 Web Gateway (in the form of a web gateway node)

The iris-key-secret is an an object of kind secret. Here we will store our key. To create it:

kubectl create secret generic iris-key-secret --from-file=iris.key

Note that you'll get an error if your file is not named iris.key. If you insist on naming it something else you can do this:

kubectl create secret generic iris-key-secret --from-file=iris.key=yourKeyFile.key

The iris-cpf is a configuration file. We will create it as an object of configmap kind.

kubectl create cm iris-cpf --from-file common.cpf

In the common.cpf file there is just the password hash. You can create it using the passwordhash image as follows:

$ docker run --rm -it containers.intersystems.com/intersystems/passwordhash:1.1 -algorithm SHA512 -workfactor 10000
Enter password:
Enter password again:
PasswordHash=2b679c8c944e2cbc2c5e4b12c62b76d5dee07f28099083940b816197ca0ffbd807c36cef7d16e17bdfe4f7a2cd45a09f6e50bef1bac8f5978362eef7d2997f3a,eac33175d6268d7bb89edb48600a3fd59d9ccd4777959bbbcc31cdb726f9b956e31fedd44c016a48d0098ffc605ac6a17b5767bfdebefe01b078ef2efd40f84f,10000,SHA512

Then put the output in your common.cpf (attached). Note that the data.cpf and compute.cpf mentioned in the IKO docs are to specify additional configuration of the data and compute nodes. This is overkill for us right now - just know that they exist.

We just want to define a password of our own at startup. If we do not, we will be prompted to change our password the first time we sign in (note that the first time the default username/password is _SYSTEM/SYS, in case you do not define one).

Onto the next secret, the one for pulling the image from the registry. I use the InterSystems Container Registry (ICR), but lots of our clients have their own registries where they push our images to. That is great too. Just note that how you create your secret depends on how you access your registry. For the ICR it is as follows:

kubectl create secret docker-registry intersystems-pull-secret --docker-server=https://containers.intersystems.com --docker-username='<your username>' --docker-password='<your password>' --docker-email='<your email>'

We have one secret left, but let's just gloss over the topology first.

Topology is the IRIS environment we want to create. Specifically, this is the data node and web gateway. Regarding the image, I see some people like to use the :latest tag as is normally good practice to ensure the most up to date software. I think in this case it would actually be better practice to specify what version one wants as it is best practice to specify the compatibilityVersion. See more about that here.

As for the webgateway, we can configure how many we want, what application paths should be available and the loginSecret. This secret is how the webgateway will be logging into IRIS.

kubectl create secret generic iris-webgateway-secret --from-literal='username=CSPSystem' --from-literal='password=SYS'

That's our last secret, but you can read up more about them on the Kubernetes documentation.

Finally, we have the serviceTemplate.

Our process will create two services that are of significance to us (the rest are outside the scope of this article and should not concern you at this time): 1) simple and 2) simple-webgateway.

For now, all you need to know about services is that they expose applications that run on pods. By running kubectl get svc, you can see external IP that these two services create. If you're running your kubernetes cluster on docker-desktop like me, then it will be localhost.

And we notice the familiar ports.

That's because this is our internal and external webservers. For example, we can go to our management portal through the external web server: http://localhost/csp/sys/UtilHome.csp. http takes us automatically to port 80 (https to 443) which is why we don't need to specify the port here.

That's it for now. In the next article we'll take another bite out of services.
 

1 Comment
讨论 (1)1
登录或注册以继续
文章
· 三月 3, 2024 阅读大约需 5 分钟

How to send messages to Microsoft Teams

Hi community,

The aim of this article is to explain how to create messaging between IRIS and Microsoft Teams.

In my company, we wanted to monitor error messages, and we used the Ens.Alert class to redirect those error messages through a Business Operation that sent an email.
The problem was that we sent those error messages to a support account where there were many emails. We wanted something specific for a specific team.

So we investigated how to make these messages reach the development team directly and they could have, in real time, a notification of an error in our production.
In our company we use Microsoft Teams as a corporate tool, so we asked ourselves: How could we make these messages reach the IRIS development team?

Previous steps

Please, expand to know how to configure your teams with the app Incoming Webhook.

 
Previous steps

Note: Webhook link is divided in two parts. Server and URL, remember this when you going to configure the component.

https://YOURCOMPANY.webhook.office.com/webhookb2/40cc6704-1bc5-4f87-xxxx-xxxxxxxxf@5xxxxxa-643b-47a3-xxxxx-fc962cc7cdb2/IncomingWebhook/6f272d796f1844b8b0b57b61365f8961/2ff46079-ee4a-442b-a642-dc418f6c67ee
Server: YOURCOMPANY.webhook.office.com
URL: /webhookb2/40cc6704-1bc5-4f87-xxxx-xxxxxxxxf@5xxxxxa-643b-47a3-xxxxx-fc962cc7cdb2/IncomingWebhook/6f272d796f1844b8b0b57b61365f8961/2ff46079-ee4a-442b-a642-dc418f6c67ee

Calling to webhook API

The Incoming Webhook app admits the Office 360 connector cards. You can create your card using the adaptivecard designer.

So, I've designed a card to display a error message (Ens.AlertRequest).

 
AdaptiveCard for Ens.AlertRequest

Using this schema, You can create the message using the messages of St.Teams like this

set class=##class(St.Teams.Msg.Adaptive.Request).%New()
set class.Type = "message"
set attach = ##class(St.Teams.Msg.Adaptive.Attachment).%New()
set content = ##class(St.Teams.Msg.Adaptive.Content).%New()

set container = ##class(St.Teams.Msg.Common.Item).%New()
set container.Type = "Container"
set item1=##class(St.Teams.Msg.Common.Item).%New()
set item1.Type = "TextBlock"
set item1.Text = "Unhandled error"
set item1.Weight = "bolder"
set item1.Size = "Medium"
set item2=##class(St.Teams.Msg.Common.Item).%New()
set item2.Type = "TextBlock"
set item2.Text = "St.Teams.BO.MainProcess"
set item2.Weight = "bolder"
set item2.Size = "small"
set item2.IsSubtitle = 1
set item3=##class(St.Teams.Msg.Common.Item).%New()
set item3.Type = "TextBlock"
set item3.Text = "ERROR <Ens>ErrFTPListFailed: 'Unable to open data connection to 127.0.0. on port 8080',código=425)"
set item3.Wrap = 1
set factSet=##class(St.Teams.Msg.Common.Item).%New()
set factSet.Type = "FactSet"
set factItem1 =##class(St.Teams.Msg.Common.FactItem).%New()
set factItem1.Title = "SessionId"
set factItem1.Value = "111"
set factItem2 =##class(St.Teams.Msg.Common.FactItem).%New()
set factItem2.Title = "Time"
set factItem2.Value = "2024-02-28 11:00:15"
do factSet.Facts.Insert(factItem1)
do factSet.Facts.Insert(factItem2)

do container.Items.Insert(item1)
do container.Items.Insert(item2)
do container.Items.Insert(item3)
do container.Items.Insert(factSet)

do content.Body.Insert(container)
set attach.Content = content
do class.Attachments.Insert(attach)

it creates the Json to call to the Webhook. But we want to create the message from a Ens.AlertRequest message, the best way is using a Data Transformer.

Then, the rule of your Ens.Alert should be like this:

It transform the Ens.AlertRequest using the St.Teams.DT.EnsAlertToAdpativeRequest and send it to St.Teams.BO.Api.Teams.

Then you recive the message directly into your Teams group.

I hope it is as useful to you as it has been to us.

15 Comments
讨论 (15)4
登录或注册以继续
文章
· 三月 2, 2024 阅读大约需 4 分钟

IKO - Lessons Learned (Part 1 - Helm)

The IKO documentation is robust. A single web page, that consists of about 50 actual pages of documentation. For beginners that can be a bit overwhelming. As the saying goes: how do you eat an elephant? One bite at a time. Let's start with the first bite: helm.

What is Helm?

Helm is to Kubernetes what the InterSystems Package Manager (IPM, formerly ObjectScript Package Manager - ZPM) is to IRIS.

It facilitates the installation of applications on the platform - in a fashion suitable for Kubernetes. That's to say that it is developed in such a way to facilitate installation to your needs, whether it be a development, test, or production environment.

We provide on our WRC software distribution all you will need under the IRIS Components tab - it consists of a .tar.gz. Extract it and you will get a .tar. Extract it again and you will see a folder iris_operator_<yourversion>. In here are a README with instructions, as well as 3 folders - an image of the IKO (you could have also got this from the InterSystems Container Registry), chart, and samples. Samples is just to help you form your files but is not actually necessary for IKO installation. Chart, however, is necessary. Let's take a peek.

chart
|
|-> iris-operator
               |
               | -> README.md
               | -> .helmignore
               | -> Chart.yaml
               | -> values.yaml
               | -> templates 
                      | -> _helpers.tpl
                      | -> apiregistration.yaml
                      | -> appcatalog-user-roles.yaml
                      | -> cleaner.yaml
                      | -> cluster-role.yaml
                      | -> cluster-role-binding.yaml
                      | -> deployment.yaml
                      | -> mutating-webhook.yaml
                      | -> NOTES.txt
                      | -> service.yaml
                      | -> service-account.yaml
                      | -> user-roles.yaml
                      | -> validating-webhook.yaml
               

 

This is the meat and potatoes (a funny way to say basic ingredients) of the application we will be installing. Don't worry. The only thing that we care about is going to be the values.yaml. Everything else is going on behind the scenes, thanks to Helm. Phew! But it's important to know that though our operator may seem like an ordinary pod, it is a lot more than that.

Most of the contents of the values.yaml are also going to be out of the scope of this article because you will not have to worry about them. We will care about just 4 fields (okay, 5 at most).

They are operator.registry, operator.repository, operator.tag, imagePullSecrets.name[0], and imagePullPolicy.

Where is your IKO image? Is your organization using a private repository? Are you planning on pulling from the ICR? Specify your image details in the registry, repository, and tag fields. If you are using the ICR you can leave it as is.

How will you access the ICR, or your organization repository? Assuming it is private you will need to specify your details with which you can access it for pulling. In the next article I touch on how to create this secret, which we can call intersystems-pull-secret instead of the standard dockerhub-secret which is what is presently there if you downloaded the files from the WRC.

Finally for the imagePullPolicy we can leave it as Always, or alternatively change it to IfNotPresent or Never. I'll refer you to the Kubernetes documentation if you need clarification - here. I tend to use IfNotPresent.

Looks like we're good to go (assuming you already have helm installed, if not install it first)! Let's install the IKO. We are going to need to tell helm where the folder with all our goodies is (that's the iris-operator folder you see above). If we were to be sitting at the chart directory you can use the command

helm install intersystems iris-operator

but perhaps you're sitting a little higher. No problem. This is fine too assuming you are sitting in a repository with iris_operator_amd-3.6.7.100:

helm install intersystems iris_operator_amd-3.6.7.100/chart/iris-operator

You'll get a message that the installation was a success and you can double check your deployment is running as is noted by the message and in our docs.

kubectl --namespace=default get deployments -l "release=intersystems, app=iris-operator"

In the next post we'll put the InterSystems Kubernetes Operator to use.

2 Comments
讨论 (2)2
登录或注册以继续
问题
· 三月 1, 2024

POST request with paging FHIR bundle

Hi everyone,

I'm looking for a strategy for dividing a large FHIR message, in a post request, into smaller parts. 

I have found the paging modifier for the GET request, but not a similiar one for the POST request. Maybe the 'batch' type of a Bundle could help me to indicate this aim but there aren't any attribute to say the total or the i-th element. 

Do you know of any method for implementing 'paging' in a post request?

2 Comments
讨论 (2)2
登录或注册以继续
文章
· 二月 29, 2024 阅读大约需 4 分钟

Testing Columnar Storage

As most of you probably already know, since approximately the end of 2022 InterSystems IRIS included the columnar storage functionality to its database, well, in today's article we are going to put it to the test in comparison to the usual row storage.

Columnar Storage

What is the main characteristic of this type of storage? Well, if we consult the official documentation we will see this fantastic table that explains the main characteristics of both types of storage (by rows or by columns):

As you can see, columnar storage is designed primarily for analytical tasks in which queries are launched against specific fields in our table, while row storage is more optimal when a large number of insertion, update and deletion operations are required. as well as obtaining complete records.

If you continue reading the documentation you will see how simple it is to configure our table to be able to use columnar storage:

CREATE TABLE table (column type, column2 type2, column3 type3) WITH STORAGETYPE = COLUMNAR

Using this command we would be defining all the columns of our table with columnar storage, but we could opt for a mixed model in which our table has row storage but certain columns make use of columnar storage.

This mixed scenario could be interesting in cases where aggregation operations such as sums, averages, etc. are common. For this case we could define which column is the one that will use said storage:

CREATE TABLE table (column type, column2 type2, column3 type3 WITH STORAGETYPE = COLUMNAR)

In the previous example we defined a table with row storage and a column (column3) with columnar storage.

Comparative

To compare the time spent by column storage and row storage in different queries, we have created a small exercise using Jupyter Notebook that will insert a series of records that we will generate in two tables, the first with storage with rows ( Test.PurchaseOrderRow) and the second with columnar storage in two of its columns (Test.PurchaseOrderColumnar)

Test.PurchaseOrderRow

CREATE TABLE Test.PurchaseOrderRow (
    Reference INTEGER,
    Customer VARCHAR(225),
    PaymentDate DATE,
    Vat NUMERIC(10,2),
    Amount NUMERIC(10,2),
    Status VARCHAR(10))

Test.PurchaseOrderColumnar

CREATE TABLE Test.PurchaseOrderColumnar (
    Reference INTEGER,
    Customer VARCHAR(225),
    PaymentDate DATE,
    Vat NUMERIC(10,2),
    Amount NUMERIC(10,2) WITH STORAGETYPE = COLUMNAR,
    Status VARCHAR(10) WITH STORAGETYPE = COLUMNAR)

If you download the Open Exchange project and deploy it in your local Docker, you can access the Jupyter Notebook instance and review the file PerformanceTests.ipynb, which will be responsible for generating the random data that we are going to store in different phases in our tables and finally it will show us a graph with the performance of the query operations.

Let's take a quick look at our project configuration:

docker-compose.yml

version: '3.7'
services:
  # iris
  iris:
    init: true
    container_name: iris
    build:
      context: .
      dockerfile: iris/Dockerfile
    ports:
      - 52774:52773
      - 51774:1972
    volumes:
    - ./shared:/shared
    environment:
    - ISC_DATA_DIRECTORY=/shared/durable
    command: --check-caps false --ISCAgent false
  # jupyter notebook
  jupyter:
    build:
      context: .
      dockerfile: jupyter/Dockerfile
    container_name: jupyter
    ports:
      - "8888:8888"
    environment:
      - JUPYTER_ENABLE_LAB=yes
      - JUPYTER_ALLOW_INSECURE_WRITES=true
    volumes:
      - ./jupyter:/home/jovyan
      - ./data:/app/data
    command: "start-notebook.sh --NotebookApp.token='' --NotebookApp.password=''" 

We deploy the IRIS and Jupyter containers in our docker, initially configuring IRIS with the namespace "TEST" and the two tables required for the test.

To avoid boring you with code, you can consult the PerformanceTests.ipynb file from which we will connect to IRIS, generate the records to be inserted and store them in IRIS

Test execution

The results have been the following (in seconds):

Inserts:

The insertions made are of bulk type:

INSERT INTO Test.PurchaseOrderColumnar (Reference, Customer, PaymentDate, Vat, Amount, Status) VALUES (?, ?, ?, ?, ?, ?)

And the time for each batch of inserts is as follows:

Total inserts

Row storage Mixed storage
1000

0.031733

0.041677

5000

0.159338

0.185252

20000

0.565775

0.642662

50000

1.486459

1.747124

100000

2.735016

3.265492

200000

5.395032

6.382278

Selects:

The Select launched includes an aggregation function and a condition, both on columns with columnar storage:

SELECT AVG(Amount) FROM Test.PurchaseOrderColumnar WHERE Status = 'SENT'

Total rows

Row storage Mixed storage
1000

0.002039

0.001178

5000

0.00328

0.000647

20000

0.005493

0.001555

50000

0.016616

0.000987

100000

0.036112

0.001605

200000

0.070909

0.002738

Conclusions

As you can see in the results obtained, the operation is exactly what is indicated in the documentation. Including columns with columnar storage has slightly penalized performance during insert (about 18% slower for our example) while queries on those same columns have dramatically improved response time (258 times faster).

It is undoubtedly something to take into account when planning the development of any application.

4 Comments
讨论 (4)1
登录或注册以继续