Feed aggregator

How to Secure all of Your Oracle Databases - Part 1

Pete Finnigan - 53 min 10 sec ago
How do you know how secure your Oracle databases are? How secure should your Oracle databases be? These are interesting questions that we will cover in this three part post. This first part is going to cover the high level....[Read More]

Posted by Pete On 19/02/24 At 01:43 PM

Categories: Security Blogs

AI not I

Greg Pavlik - 4 hours 29 min ago

The notion that what we call AI is somehow approaching a form on consciousness remains an absurdity: fantastical thinking by people who really ought to spend a minimal amount of time at least reading up on philosophy of mind. Generative AI fits perfectly into John Searle's Chinese Room (the main variation is probability replaces rules, which reflects the one major innovation of NLP over decades).

I don't mean to suggest the technology is not extremely useful - it is, and will become more so. But: reality check.

Oracle ZDM Migration – java.security.InvalidKeyException: invalid key format

Yann Neuhaus - Sun, 2024-02-18 11:04

ZDM tool migration requires SSH Passwordless Login without passphrase between ZDM Host, the source and the target. Configuring appropriate keys might still result in a java security exception on this one. In this blog I will tell you how to deal with such a problem. I faced this problem implementing ZDM to migrate On-Premise Database to new ExaCC at one of our customer.

Read more: Oracle ZDM Migration – java.security.InvalidKeyException: invalid key format Setting up SSH Passwordless Login

First of all we need to create the private and public key on the ZDM Host.

From the ZDM host, with zdmuser, go in the ~/.ssh folder and run ssh-keygen.

[zdmuser@zdmhost .ssh]$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/zdmuser/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/zdmuser/.ssh/id_rsa.
Your public key has been saved in /home/zdmuser/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:8uTp************************ziw zdmuser@zdmhost
The key's randomart image is:
+---[RSA 3072]----+
|   oo+==.        |
...
...
...
|    o.+..        |
+----[SHA256]-----+

This will create 2 keys, one private (id_rsa) and one public (id_rsa_pub).

Get the content of the public key.

[zdmuser@zdmhost .ssh]$ cat id_rsa.pub
ssh-rsa AAAA************************vaU= zdmuser@zdmhost

And add the content of the public RSA key to the authorized_keys file from both ExaCC Cluster VMs (target ExaCC-cl01n1 and ExaCC-cl01n2) opc user and the on-premises VM (source vmonpr) oracle user.

[opc@ExaCC-cl01n1 .ssh]$ echo "ssh-rsa AAAA************************vaU= zdmuser@zdmhost" >> authorized_keys

[opc@ExaCC-cl01n2 .ssh]$ echo "ssh-rsa AAAA************************vaU= zdmuser@zdmhost" >> authorized_keys

oracle@vmonpr:/home/oracle/.ssh/ [ONPR] echo "ssh-rsa AAAA************************vaU= zdmuser@zdmhost" >> authorized_keys

We will then test SSH connection to the 3 VMs and ensure no password are requested. Example:

[zdmuser@zdmhost migration]$ ssh opc@ExaCC-cl01n1
Last login: Fri Feb  2 16:58:04 2024 from 10.160.52.122
[opc@ExaCC-cl01n1 ~]$

Check ZDM migration

Checking ZDM migration with zdmcli and -eval option might get failed:

[zdmuser@zdmhost migration]$ /u01/app/oracle/product/zdm/bin/zdmcli migrate database -sourcesid ONPR -rsp /home/zdmuser/migration/zdm_ONPR_physical_online.rsp -sourcenode vmonpr -srcauth zdmauth -srcarg1 user:oracle -srcarg2 identity_file:/home/zdmuser/.ssh/id_rsa -srcarg3 sudo_location:/usr/bin/sudo -targetnode ExaCC-cl01n1 -tgtauth zdmauth -tgtarg1 user:opc -tgtarg2 identity_file:/home/zdmuser/.ssh/id_rsa -tgtarg3 sudo_location:/usr/bin/sudo -tdekeystorepasswd -tgttdekeystorepasswd -eval
zdmhost.domain.com: Audit ID: 50
Enter source database ONPR SYS password:
zdmhost: 2024-02-02T16:30:19.487Z : Processing response file ...
Operation "zdmcli migrate database" scheduled with the job ID "11".

[zdmuser@zdmhost migration]$ /u01/app/oracle/product/zdm/bin/zdmcli query job -jobid 11
zdmhost.domain.com: Audit ID: 52
Job ID: 11
User: zdmuser
Client: zdmhost
Job Type: "EVAL"
Scheduled job command: "zdmcli migrate database -sourcesid ONPR -rsp /home/zdmuser/migration/zdm_ONPR_physical_online.rsp -sourcenode vmonpr -srcauth zdmauth -srcarg1 user:oracle -srcarg2 identity_file:/home/zdmuser/.ssh/id_rsa -srcarg3 sudo_location:/usr/bin/sudo -targetnode ExaCC-cl01n1 -tgtauth zdmauth -tgtarg1 user:opc -tgtarg2 identity_file:/home/zdmuser/.ssh/id_rsa -tgtarg3 sudo_location:/usr/bin/sudo -tdekeystorepasswd -tgttdekeystorepasswd -eval"
Scheduled job execution start time: 2024-02-02T17:30:19+01. Equivalent local time: 2024-02-02 17:30:19
Current status: FAILED
Result file path: "/u01/app/oracle/chkbase/scheduled/job-11-2024-02-02-17:30:48.log"
Metrics file path: "/u01/app/oracle/chkbase/scheduled/job-11-2024-02-02-17:30:48.json"
Job execution start time: 2024-02-02 17:30:48
Job execution end time: 2024-02-02 17:30:48
Job execution elapsed time: 0 seconds

Result file "/u01/app/oracle/chkbase/scheduled/job-11-2024-02-02-17:30:48.log" contents:
zdmhost: 2024-02-02T16:30:48.591Z : Processing response file ...
zdmhost: 2024-02-02T16:30:48.595Z : Processing response file ...
PRCZ-4002 : failed to execute command "/bin/cp" using the privileged execution plugin "zdmauth" on nodes "ExaCC-cl01n1"
java.security.InvalidKeyException: invalid key format

Error of failed execution is :

java.security.InvalidKeyException: invalid key format
Solution

The problem is due to the fact that ZDM only supports RSA key and the generated key was an OPENSSH key.

Checking current key, we can see that the key is an openssh key:

[zdmuser@zdmhost .ssh]$ head -n1 id_rsa
-----BEGIN OPENSSH PRIVATE KEY-----

[zdmuser@zdmhost .ssh]$ tail -n 1 id_rsa
-----END OPENSSH PRIVATE KEY-----

We need to convert the private key to PEM format.

[zdmuser@zdmhost .ssh]$ ssh-keygen -p -m PEM -f ~/.ssh/id_rsa
Key has comment 'zdmuser@zdmhost'
Enter new passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved with the new passphrase.

The new key looks now like.

[zdmuser@zdmhost .ssh]$ head -n1 id_rsa
-----BEGIN RSA PRIVATE KEY-----

[zdmuser@zdmhost .ssh]$ tail -n 1 id_rsa
-----END RSA PRIVATE KEY-----

And, now, zdmcli eval command is succeeding.

[zdmuser@zdmhost migration]$ /u01/app/oracle/product/zdm/bin/zdmcli query job -jobid 39
zdmhost.domain.com: Audit ID: 434
Job ID: 39
User: zdmuser
Client: zdmhost
Job Type: "EVAL"
Scheduled job command: "zdmcli migrate database -sourcesid ONPR -rsp /home/zdmuser/migration/zdm_ONPR_physical_online.rsp -sourcenode vmonpr -srcauth zdmauth -srcarg1 user:oracle -srcarg2 identity_file:/home/zdmuser/.ssh/id_rsa -srcarg3 sudo_location:/usr/bin/sudo -targetnode ExaCC-cl01n1 -tgtauth zdmauth -tgtarg1 user:opc -tgtarg2 identity_file:/home/zdmuser/.ssh/id_rsa -tgtarg3 sudo_location:/usr/bin/sudo -tdekeystorepasswd -tgttdekeystorepasswd -eval"
Scheduled job execution start time: 2024-02-14T14:18:19+01. Equivalent local time: 2024-02-14 14:18:19
Current status: SUCCEEDED
Result file path: "/u01/app/oracle/chkbase/scheduled/job-39-2024-02-14-14:18:29.log"
Metrics file path: "/u01/app/oracle/chkbase/scheduled/job-39-2024-02-14-14:18:29.json"
Job execution start time: 2024-02-14 14:18:29
Job execution end time: 2024-02-14 14:21:18
Job execution elapsed time: 2 minutes 48 seconds
ZDM_GET_SRC_INFO ........... PRECHECK_PASSED
ZDM_GET_TGT_INFO ........... PRECHECK_PASSED
ZDM_PRECHECKS_SRC .......... PRECHECK_PASSED
ZDM_PRECHECKS_TGT .......... PRECHECK_PASSED
ZDM_SETUP_SRC .............. PRECHECK_PASSED
ZDM_SETUP_TGT .............. PRECHECK_PASSED
ZDM_PREUSERACTIONS ......... PRECHECK_PASSED
ZDM_PREUSERACTIONS_TGT ..... PRECHECK_PASSED
ZDM_VALIDATE_SRC ........... PRECHECK_PASSED
ZDM_VALIDATE_TGT ........... PRECHECK_PASSED
ZDM_POSTUSERACTIONS ........ PRECHECK_PASSED
ZDM_POSTUSERACTIONS_TGT .... PRECHECK_PASSED
ZDM_CLEANUP_SRC ............ PRECHECK_PASSED
ZDM_CLEANUP_TGT ............ PRECHECK_PASSED

L’article Oracle ZDM Migration – java.security.InvalidKeyException: invalid key format est apparu en premier sur dbi Blog.

Alfresco – A never ending transformation

Yann Neuhaus - Fri, 2024-02-16 10:08

Beginning of the week, as I was working for our ServiceDesk (SLA support for our customers), I saw a few dozen mails generated by our monitoring over the weekend on a Production Alfresco 7.x Cluster doing the yo-yo in terms of RAM and Disk Space. Nothing was down, just some strange behavior where 20GB of free space would be gone and then re-appear after a few minutes and same thing for the RAM/SWAP.

The first thing I checked was the disk space mentioned on the alert. We received alerts from all members of the cluster one by one, almost in a perfect round-robin manner. On the second node, I saw the issue occurring in real-time, so I looked into what exactly was generating all the noise:

alfresco@alf-p2:~# date; df -h /tmp
Mon Feb 12 07:27:41 UTC 2024
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb2        19G    7G   12G  35% /tmp
alfresco@alf-p2:~#
alfresco@alf-p2:~# date; df -h /tmp
Mon Feb 12 07:28:20 UTC 2024
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb2        19G    9G    9G  49% /tmp
alfresco@alf-p2:~#
alfresco@alf-p2:~# du -sm /tmp/
9427    /tmp/
alfresco@alf-p2:~#
alfresco@alf-p2:~# du -sm /tmp/
9484    /tmp/
alfresco@alf-p2:~#
alfresco@alf-p2:~# du -sm /tmp/
9541    /tmp/
alfresco@alf-p2:~#

In less than a minute, around 2/3Gb of temporary files were generated, which doesn’t seem very healthy:

alfresco@alf-p2:~# cd /tmp
alfresco@alf-p2:/tmp#
alfresco@alf-p2:/tmp# ls -ltr
total 480
...
-rw-r-----   1 alfresco  alfresco    115 Feb 11 21:26 scheduler.json
drwxr-x---   2 alfresco  alfresco   4096 Feb 12 07:28 Alfresco/
drwxrwxrwt 117 root      root      12288 Feb 12 07:28 ./
alfresco@alf-p2:/tmp#
alfresco@alf-p2:/tmp# cd Alfresco/
alfresco@alf-p2:/tmp/Alfresco# ls -l
total 10553428
drwxr-x---   2 alfresco alfresco        4096 Feb 12 07:29 ./
drwxrwxrwt 117 root     root           12288 Feb 12 07:29 ../
-rw-r-----   1 alfresco alfresco     1897650 Feb 12 07:23 source_11877384286747332767_tmp.pdf
-rw-r-----   1 alfresco alfresco 10804789248 Feb 12 07:29 target_18121744399232974935_tmp.txt
alfresco@alf-p2:/tmp/Alfresco#
alfresco@alf-p2:/tmp/Alfresco#
alfresco@alf-p2:/tmp/Alfresco# ls -l
total 10686460
drwxr-x---   2 alfresco alfresco        4096 Feb 12 07:29 ./
drwxrwxrwt 117 root     root           12288 Feb 12 07:29 ../
-rw-r-----   1 alfresco alfresco     1897650 Feb 12 07:23 source_11877384286747332767_tmp.pdf
-rw-r-----   1 alfresco alfresco 10941014016 Feb 12 07:29 target_18121744399232974935_tmp.txt
alfresco@alf-p2:/tmp/Alfresco#

At that point in time, it looked like Alfresco was doing something that was causing the issue for the Disk Space, at least. Here, we can see a PDF file that is a “source” and a TXT file that appears to be under generation, as a “target”. So of course, my first thought here is that this is probably the Alfresco Transformation Service that is causing this issue, trying to transform a PDF into TXT, most probably for indexing of the content of this file.

While looking at the RAM/SWAP usage on this server, it was also showing the same thing, with the Java process of the ATS using 100% CPU (fortunately, the host has multiple CPUs) and going overboard with its RAM, forcing the host to SWAP.

Therefore, I looked at the ATS logs and saw 2 types of errors. First was a few IOException on PDFBox “Error: End-Of-File: expected line” but there wasn’t a lot of those… Then there was another error, much more present, that was the consequence of the FileSystem being full:

alfresco@alf-p2:~# cat $ATS_HOME/logs/transform-core-aio.log
...
2024-02-12 07:18:37.380 ERROR 23713 --- [o-8090-exec-141] o.a.transformer.TransformController      : Error writing: Seite 1

org.alfresco.transform.exceptions.TransformException: Error writing: Seite 1
        at org.alfresco.transformer.executors.Transformer.transform(Transformer.java:83) ~[alfresco-transformer-base-2.5.3.jar!/:2.5.3]
        at org.alfresco.transformer.AIOController.transformImpl(AIOController.java:118) ~[classes!/:2.5.3]
        at org.alfresco.transformer.AbstractTransformerController.transform(AbstractTransformerController.java:173) ~[alfresco-transformer-base-2.5.3.jar!/:2.5.3]
        at jdk.internal.reflect.GeneratedMethodAccessor75.invoke(Unknown Source) ~[na:na]
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
        at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
        ...
Caused by: java.lang.IllegalStateException: Error writing: Seite 1
        at org.alfresco.transformer.executors.Tika.transform(Tika.java:697) ~[alfresco-transform-tika-2.5.3.jar!/:2.5.3]
        at org.alfresco.transformer.executors.Tika.transform(Tika.java:673) ~[alfresco-transform-tika-2.5.3.jar!/:2.5.3]
        at org.alfresco.transformer.executors.Tika.transform(Tika.java:617) ~[alfresco-transform-tika-2.5.3.jar!/:2.5.3]
        at org.alfresco.transformer.executors.TikaJavaExecutor.call(TikaJavaExecutor.java:141) ~[alfresco-transform-tika-2.5.3.jar!/:2.5.3]
        at org.alfresco.transformer.executors.TikaJavaExecutor.transform(TikaJavaExecutor.java:131) ~[alfresco-transform-tika-2.5.3.jar!/:2.5.3]
        at org.alfresco.transformer.executors.Transformer.transform(Transformer.java:70) ~[alfresco-transformer-base-2.5.3.jar!/:2.5.3]
        ... 55 common frames omitted
Caused by: org.xml.sax.SAXException: Error writing: Seite 1
        at org.apache.tika.sax.ToTextContentHandler.characters(ToTextContentHandler.java:110) ~[tika-core-1.26.jar!/:1.26]
        at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146) ~[tika-core-1.26.jar!/:1.26]
        at org.apache.tika.sax.WriteOutContentHandler.characters(WriteOutContentHandler.java:136) ~[tika-core-1.26.jar!/:1.26]
        at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146) ~[tika-core-1.26.jar!/:1.26]
        ...
        at org.alfresco.transformer.executors.Tika.transform(Tika.java:693) ~[alfresco-transform-tika-2.5.3.jar!/:2.5.3]
        ... 60 common frames omitted
        Suppressed: java.io.IOException: No space left on device
                at java.base/java.io.FileOutputStream.writeBytes(Native Method) ~[na:na]
                at java.base/java.io.FileOutputStream.write(FileOutputStream.java:354) ~[na:na]
                at java.base/sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:233) ~[na:na]
                at java.base/sun.nio.cs.StreamEncoder.implClose(StreamEncoder.java:337) ~[na:na]
                at java.base/sun.nio.cs.StreamEncoder.close(StreamEncoder.java:161) ~[na:na]
                at java.base/java.io.OutputStreamWriter.close(OutputStreamWriter.java:255) ~[na:na]
                at java.base/java.io.BufferedWriter.close(BufferedWriter.java:269) ~[na:na]
                at org.alfresco.transformer.executors.Tika.transform(Tika.java:684) ~[alfresco-transform-tika-2.5.3.jar!/:2.5.3]
                ... 60 common frames omitted
Caused by: java.io.IOException: No space left on device
        at java.base/java.io.FileOutputStream.writeBytes(Native Method) ~[na:na]
        at java.base/java.io.FileOutputStream.write(FileOutputStream.java:354) ~[na:na]
        at java.base/sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:233) ~[na:na]
        at java.base/sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:303) ~[na:na]
...
alfresco@alf-p2:~#

As you can see above, at 07:18, the FileSystem /tmp was 100% full and when I checked 5 minutes later, at 07:23, a new transformation was already producing a 10Gb text file and still growing. So, it was clear that this happens repeatedly, most probably for the same document. According to the monitoring, the issue started just before the weekend. Looking at the first occurrences of the FileSystem full from the ATS logs gave the following:

alfresco@alf-p2:~# grep '2024.*Error writing' $ATS_HOME/logs/transform-core-aio.log
2024-02-09 19:20:51.628 ERROR 23713 --- [o-8090-exec-166] o.a.transformer.TransformController      : Error writing:
2024-02-09 19:41:29.954 ERROR 23713 --- [o-8090-exec-156] o.a.transformer.TransformController      : Error writing: Seite 1
2024-02-09 20:02:11.764 ERROR 23713 --- [o-8090-exec-160] o.a.transformer.TransformController      : Error writing: Seite 1
2024-02-09 20:23:08.828 ERROR 23713 --- [o-8090-exec-163] o.a.transformer.TransformController      : Error writing:
2024-02-09 20:44:05.313 ERROR 23713 --- [o-8090-exec-141] o.a.transformer.TransformController      : Error writing: Seite 1
2024-02-09 21:04:52.642 ERROR 23713 --- [o-8090-exec-162] o.a.transformer.TransformController      : Error writing: Seite 1
...
2024-02-12 07:18:37.380 ERROR 23713 --- [o-8090-exec-152] o.a.transformer.TransformController      : Error writing: Seite 1
alfresco@alf-p2:~#

With the above, it pretty much confirms that it’s the same document that is always failing, since it’s blocking on “Seite 1“, which means “Page 1” in English.

To be able to find which document is causing the issue in Alfresco, there isn’t a lot of details available, since the ATS isn’t really giving you much about what it is doing. All I had was a temporary name (which obviously doesn’t trace back to anything in the Repository) and a size. Therefore, I checked for documents on the Alfresco Data (“alf_data“) with a size equal to the document “/tmp/Alfresco/source_11877384286747332767_tmp.pdf” (i.e. 1897650 bytes), created in the last few days. I expected it to be created on the 9-Feb, a little before 19:20 and I indeed found one:

alfresco@alf-p2:~# find /alf_data/contentstore/2024/2/ -type f -ls | grep 1897650
 34508512  1856 -rw-r----- 1 alfresco alfresco 1897650 Feb 9 19:02 /alf_data/contentstore/2024/2/9/19/02/174f569e-93a3-4829-8ad5-bd3d6e78447b.bin
alfresco@alf-p2:~#
alfresco@alf-p2:~# md5sum /tmp/Alfresco/source_11877384286747332767_tmp.pdf /alf_data/contentstore/2024/2/9/19/02/174f569e-93a3-4829-8ad5-bd3d6e78447b.bin
45ed40bd5f84b7c68e246885f2b6a55f  /tmp/Alfresco/source_11877384286747332767_tmp.pdf
45ed40bd5f84b7c68e246885f2b6a55f  /alf_data/contentstore/2024/2/9/19/02/174f569e-93a3-4829-8ad5-bd3d6e78447b.bin
alfresco@alf-p2:~#
alfresco@alf-p2:~# diff /tmp/Alfresco/source_11877384286747332767_tmp.pdf /alf_data/contentstore/2024/2/9/19/02/174f569e-93a3-4829-8ad5-bd3d6e78447b.bin
alfresco@alf-p2:~#

Therefore, this is the same content file. There is of course the possibility that a duplicate node was using the same content before February (as I searched only inside /2024/2, that means February), but since the issue appeared only over the weekend, it’s pretty safe to assume it’s this document/node.

alfresco@alf-p2:~# stat /alf_data/contentstore/2024/2/9/19/02/174f569e-93a3-4829-8ad5-bd3d6e78447b.bin
  File: /alf_data/contentstore/2024/2/9/19/02/174f569e-93a3-4829-8ad5-bd3d6e78447b.bin
  Size: 1897650         Blocks: 3712       IO Block: 262144 regular file
Device: 34h/52d Inode: 34508512    Links: 1
Access: (0640/-rw-r-----)  Uid: (  113/alfresco)   Gid: (  116/alfresco)
Access: 2024-02-09 19:02:12.153002964 +0000
Modify: 2024-02-09 19:02:12.157983495 +0000
Change: 2024-02-09 19:02:12.157983635 +0000
 Birth: -
alfresco@alf-p2:~#

From that point, I had the “content_url” of a Node. Therefore, I could have used the Database (see useful database queries) to find the NodeRef of this Alfresco Node but at this customer, I don’t have an easy access to the DB, so I went through Share instead.

I know the node was created (or modified) at 19:02:12 (+/- 1s) on the 9-Feb, and even if the content isn’t indexed, its metadata should still be available searchable. Therefore, I just performed a search on Alfresco Share, to find documents created (or modified) at that exact time, i.e. cm:created:’2024-02-09T19:02:12′.

That gave me 4 results, out of which only 1 had a size around 2MB. To validate if this was indeed the document causing the issue, I simply used the JavaScript Console to dump this file and it gave me the exact same “content_url“. I could also validate on Share that this specific file wasn’t content-indexed yet (despite being in the repository for 2.5 days).

As a temporary workaround, to stop the OS from going crazy, I set this document as metadata-indexed only (no content), using the “Index Control” aspect. If you don’t know how this works, it’s pretty simple for a node:

  • Click on “Manage Aspect”
  • From the list of “Available to Add”, find “Index Control (cm:indexControl)”
  • Click on “+” to add it to the list of “Currently Selected”
  • Click on “Apply changes”
  • Click on “Edit Properties”
  • Uncheck the “Is Content Indexed” option

After doing that, you should be able to see something like that on the node’s properties:

Alfresco Index Control

In case a transformation for this document is already in progress, you will need to wait for the FileSystem to be full for the ATS (java) to remove its temporary file and realize that this document doesn’t need to be transformed anymore. You can probably also restart the process, if you prefer.

That’s only a workaround of course, not a real solution. Therefore, even if I knew that the issue was most probably around “Seite 1“, I replicated the issue on TEST by uploading this same file into the TEST environment and then looked inside the TXT content, to validate that assumption:

alfresco@alf-t1:/tmp/Alfresco# ls -l
total 23960
drwxr-x---  2 alfresco alfresco      4096 Feb 12 09:10 ./
drwxrwxrwt 25 root     root         36864 Feb 12 09:10 ../
-rw-r-----  1 alfresco alfresco   1897650 Feb 12 09:10 source_2995534351432950419_tmp.pdf
-rw-r-----  1 alfresco alfresco  22593536 Feb 12 09:10 target_7429882841367188802_tmp.txt
alfresco@alf-t1:/tmp/Alfresco#
alfresco@alf-t1:/tmp/Alfresco# wc -l target_7429882841367188802_tmp.txt
2509490 target_7429882841367188802_tmp.txt
alfresco@alf-t1:/tmp/Alfresco#
alfresco@alf-t1:/tmp/Alfresco# grep -v "^[[:space:]]*Seite 1$" target_7429882841367188802_tmp.txt | wc -l
1913
alfresco@alf-t1:/tmp/Alfresco#
alfresco@alf-t1:/tmp/Alfresco# sleep 30
alfresco@alf-t1:/tmp/Alfresco#
alfresco@alf-t1:/tmp/Alfresco# wc -l target_7429882841367188802_tmp.txt
83418233 target_7429882841367188802_tmp.txt
alfresco@alf-t1:/tmp/Alfresco#
alfresco@alf-t1:/tmp/Alfresco# grep -v "^[[:space:]]*Seite 1$" target_7429882841367188802_tmp.txt | wc -l
1913
alfresco@alf-t1:/tmp/Alfresco#

As shown above, there are 1913 lines of some texts and then the rest of the millions of lines are exactly “ Seite 1“. This text is actually coming from the page 34 of the PDF (it’s a merge of multiple PDFs it seems). By removing the page 34 from the document, it can be indexed properly. In the end, the “quick” solution for this customer is to fix the PDF (e.g. transform the page 34 into an image, then back into a PDF and OCRize it so it is indexed and searchable).

L’article Alfresco – A never ending transformation est apparu en premier sur dbi Blog.

An AWS journey in Geneva

Yann Neuhaus - Fri, 2024-02-16 09:18

This week I could go to the AWS re:Invent re:Cap in Geneva. This event, organized by AWS, gives a resume of the biggest announcements made in the famous re:Invent in Las Vegas.

The topics addressed during this event covered various cloud technologies where AWS is investing resources to develop. We had a recap of the news regarding Generative AI, Data and Analytics, App Modernization and Developer Experience or event Management and Governance.

Generative AI

In the generative AI part, we were mainly introduced to Amazon Q, a chat bot made in AWS. As of the features we saw that Q can integrate with more than 40 data sources. It also uses multiple sources for user management like AWS SSO or Entra ID. The main purpose is to answer user queries based on their rights to access documents, so the answers are filtered so users only what they are allowed to.

Database and Analytics

For the Database and Analytics topic we had an introduction to zero ETL. This topic shows AWS ambitions to reduce the need of pipelines. They want to create trust relationships between databases and redshift to get rid of the pipelines.

Another point presented in this topic was translator to generate SQL query based on the human language. This feature is used by Redshift and will enable people to create SQL queries based on natural phrases. It also uses Amazon Q as a base.

Applications and DevExp

During this session, we mainly learned about CodeCatalyst, which is a unified software development service. It allows us to do a full project management in one tool, with issues as Gitlab/Github. A big point here was the role of Amazon Q. We can directly assign issues to this IA tool and it will do the job to resolve it. Then we just have to commit the code, merge it and that’s it.

We also got an introduction to CodeWhisper, a tool that helps developer to create their code. This service is now compatible with IaC languages, like Terraform and it can be integrated directly into some IDEs like Visual Studio.

Management and Governance

Last but not least, we had an introduction to the newest features in terms of management and governance. In terms of security, we learned that AWS inspector can now be used with CI/CD plugins or that GuardDuty can now look at runtime events in containers.

We also had a talk about the frugal architect way int eh cloud, which are pillars created by Dr Vogel to help people have more sustainable and more performant cloud services by optimizing the costs. We were also introduce to the newest monitoring features for example a new Application panel that shows us all the costs for a specific application deployed in AWS.

Most of the features we saw were only on preview or available in some specific regions (mostly US regions). No doubt that all those features will soon be available globally.

L’article An AWS journey in Geneva est apparu en premier sur dbi Blog.

Business As Usual (BAU) vs Project Work

Tim Hall - Fri, 2024-02-16 07:10

I’ve had this conversation so many times over the years, and I’m sure I’ve written about elements of it several times, but I’m not sure I’ve written about it specifically before, so here goes… In every organisation there are conflicting demands from project work and business as usual (BAU) tasks. In case you’ve not heard … Continue reading "Business As Usual (BAU) vs Project Work"

The post Business As Usual (BAU) vs Project Work first appeared on The ORACLE-BASE Blog.Business As Usual (BAU) vs Project Work was first posted on February 16, 2024 at 2:10 pm.
©2012 "The ORACLE-BASE Blog". Use of this feed is for personal non-commercial use only. If you are not reading this article in your feed reader, then the site is guilty of copyright infringement.

Attach database error (SMB Fileshare)

Yann Neuhaus - Fri, 2024-02-16 05:48

Starting with SQL Server 2012, SQL Server supports databases using the Server Message Block (SMB) file server as a storage option. For performance and reliability reasons, SMB 3.0 or later is preferred to use.
For architectures like Always On Failover Cluster Instance (FCI), it can be a relevant alternative to cluster disks. To avoid a single point of failure (SPOF) scenario, the file shares should be high available.

Issue description

In a production context, we set up a FCI infrastructure, in Multi-AZ, on AWS using Amazon FSx as file share target – also in multi-AZ deployment to avoid SPOF scenario.
Everything run smoothly until a Detach / Attach approach was used at some point. SQL Server triggered the following error message 5120 at the Attach phase:

Msg 5120, Level 16, State 101, Line 12
Unable to open the physical file "\fsx-server\SharePath\Database.mdf". Operating system error 5: "5(Access is denied.)".
Msg 1802, Level 16, State 7, Line 12
CREATE DATABASE failed. Some file names listed could not be created. Check related errors.

Even if SQL Server service account has Full Control on the files, it cannot attach back after the detach process.
In some documentation, you can find the recommendation to enable Trace Flag 1802, which will disable ACLs change and impersonated access verification during attach & detach operations. But another error (5123) appears anyway:

Msg 5123, Level 16, State 20, Line 11
CREATE FILE encountered operating system error 5(Access is denied.) while attempting to open or create the physical file '\fsx-server\SharePath\Database.mdf'.
Detach Process

To understand the root-cause of this error, let’s dig into the Microsoft documentation regarding detaching process.
When detaching the database files, here are the following permissions applying on the files:

  • Without impersonation: the SQL Server service account and members of local Windows Administrator group (i.e. FSx related) are granted FULL control on the database files
  • With impersonation: only the account performing the operation is granted FULL control on the database files

Initially without Trace Flag 1802, we were in the first situation “without impersonation” after detaching the database:

  • ACLs inheritance is disabled on database files
  • SQL Server service account has FULL Control on the database files
  • Local Windows Administrator group of the File Server has FULL control on database files
Attach Process

Now, let’s now dig into the Attach process still from the Microsoft documentation.
When attaching database files to create the database in SQL Server, here are the following permissions which are applying to the files:

  • Without impersonation: only the SQL Server service account is added with FULL Control permissions
  • With impersonation: the SQL Server service account and members of the local Windows Administrator group are granted FULL Control permissions

The specificity of Attach process, compared to the creation of a database from scratch, it tries to apply NTFS changes on existing files. We will see this slightly difference is not insignificant.

SMB Security Requirements

Here are the requirements in the context the database files are located on an SMB file share for the SQL Server Service (domain) account:

  • FULL Control share permissions
  • FULL Control NTFS permissions on the SMB share folders
  • SeSecurityPriviledge (Local Security Policy on the File Server) is required to change get/set ACLs. This is true even if the account has FULL Control NTFS permissions on the database file
Issue root-cause

Attach process changes the current permissions of database files before creating the database in SQL Server. To do that, it requires to get/set ACLs on the files: so SeSecurityPriviledge is required at the File Server side in order to avoid error 5120 & 5123.
If you are managing the File server, this is a change you can implement and fix this issue. But in a context where SMB share is a Managed service, we were not able to fix it.

Limitations on Attach process is not blocking because it exists other approaches – such as Backup & Restore – but it can be disturbing when you are facing this issue without understanding the root-cause.
Similar to PaaS environments, Attach & Detach is not supported as soon as you have a managed service on the Operating System hosting the database files.

L’article Attach database error (SMB Fileshare) est apparu en premier sur dbi Blog.

High db block gets for inserting into reference partitioned table

Tom Kyte - Thu, 2024-02-15 01:46
Hello Tom, Could you please advise why I'm getting so huge difference in db block gets and redo for insert between range and reference partitioned table? Db block gets are like 100x more for reference partitioned table and insert is 2-3 times slower. <code> DB01> create table t1 (id number(19) primary key, ts date) 2 partition by range (ts) interval (numtodsinterval(1, 'DAY')) (partition P0001 values less than (to_date('2024-01-01' ,'YYYY-MM-DD'))); Table created. DB01> DB01> insert into t1 (id, ts) values (1, sysdate); 1 row created. DB01> DB01> DB01> -- range interval DB01> create table t2 (id number(19), t1_id number(19) not null, constraint t2_fk foreign key (t1_id) references t1 (id)) 2 partition by range (t1_id) interval (1) (partition values less than (1)); Table created. DB01> set autotrace trace exp stat DB01> insert into t2 (id, t1_id) select level, 1 from dual connect by level <= 2000000; 2000000 rows created. Execution Plan ---------------------------------------------------------- Plan hash value: 1236776825 ------------------------------------------------------------------------------ | Id | Operation | Name | Rows | Cost (%CPU)| Time | ------------------------------------------------------------------------------ | 0 | INSERT STATEMENT | | 1 | 2 (0)| 00:00:01 | | 1 | LOAD TABLE CONVENTIONAL | T2 | | | | |* 2 | CONNECT BY WITHOUT FILTERING| | | | | | 3 | FAST DUAL | | 1 | 2 (0)| 00:00:01 | ------------------------------------------------------------------------------ Predicate Information (identified by operation id): --------------------------------------------------- 2 - filter(LEVEL<=2000000) Statistics ---------------------------------------------------------- 105 recursive calls 51252 db block gets 7237 consistent gets 0 physical reads 147628492 redo size 123 bytes sent via SQL*Net to client 391 bytes received via SQL*Net from client 1 SQL*Net roundtrips to/from client 2 sorts (memory) 0 sorts (disk) 2000000 rows processed DB01> set autotrace off DB01> commit; Commit complete. DB01> DB01> DB01> -- reference DB01> create table t3 (id number(19), t1_id number(19) not null, constraint t3_fk foreign key (t1_id) references t1 (id)) 2 partition by reference (t3_fk); Table created. DB01> set autotrace trace exp stat DB01> insert into t3 (id, t1_id) select level, 1 from dual connect by level <= 2000000; 2000000 rows created. Execution Plan ---------------------------------------------------------- Plan hash value: 1236776825 ------------------------------------------------------------------------------ | Id | Operation | Name | Rows ...
Categories: DBA Blogs

Kubernetes Networking by Using Cilium – Beginner Level

Yann Neuhaus - Wed, 2024-02-14 03:37

There seems to be a strong interest these days about understanding better networking in Kubernetes. This blog post is my contribution on this topic. I’ll do my best to explain it in a visual way and translate the techie part to an understandable language so that anybody can actually enjoy it.

The best way to learn networking is by doing what is called “Follow the packet” or “The life of a packet”. Basically, you follow the packet from the sender to the receiver by stopping at each step of this journey. I did that a while ago with a communication from a Pod to another Pod by using Calico. This time I’ll use another Container Network Interface (CNI) called Cilium which is based on eBPF (understand fast and flexible routing) and comes with a lot of powerful features and tools. Let’s hit that packet road!

Traditional Networking without Kubernetes

We are going to start from the beginning. I’ll consider you know nothing about networking. Maybe you already know what an IP Address is? The IP Address is the numerical Address of a network interface in a computer. This is how your computer can connect for example to your Wi-Fi network and give you access to the Internet. If you are using a laptop, your Wi-Fi network interface has one IP Address. This network interface also has another address that is unique and burnt on it by the hardware provider. This address is called the Medium Access Control (MAC) Address.

The IP Address belongs to a group (the IP Subnet). To know to which group it belongs to, it uses something called the Subnet Mask. This mask when applied to the IP Address gives a result and this result is identical for every IP Address that belong to the same group. This is like your community.

Let’s use the drawing below to make analogies:

The house is a computer, a server or a virtual machine. It could be of different sizes according to its CPU and memory but let use the same one for simplicity sake. A house has a door that is your network interface. The serial number of that door is your MAC Address and the number on your house (which is usually nailed on the door) is your IP Address. It is only if you change your door that your serial number will change. However, your house number has been assigned to you by the architect of your community and could change if there is a reassignment or a change in the design.

The community 10th (using numbers from 10 to 19) in blue belong to the same group (same IP subnet) and the community 20th in green is another group. In each group there are five houses so there is space for the community to grow. In each community the door has a direct link to a fountain which represents a Switch. At the fountain, there is a sign for each path to indicate which door you can reach. Yes, the fountain doesn’t know the house number but only the serial number of the door. For humans it is not very practical so we use a map (called the ARP table) that gives the translation between the house number and the serial number of its door.

If you live in house 14 and want to visit house 15, you’ll use the path (there is only one and it is yours so no traffic jams!) to the fountain first and then look at the signs. You know from your map which serial number correspond to which house and so you use the path to house 15. In this star topology you always go to the fountain first and not directly to the house you want to visit because there is no direct path. A path inside your community represents a Layer 2 link. You can’t reach another community by using these paths.

Traveling between communities

What now if from your house 14, you want to pay a visit to house 24? This is another community which means the couple IP Address / Subnet mask for 14 doesn’t produce the same result as for 24. Indeed the 10th and the 20th communities are different. So you know the destination is another community and in that case you have to go first to your gatekeeper (but always through the fountain as we have seen). He is the default gateway of your community and he lives in house 11. The rule is to go to him for any destinations that is outside of your community.

Only he has a map (the Routing Table) to reach community 20th and know which road to take (This is called Layer 3 routing because you are traveling outside of your community). This map shows that to reach 24 in the 20th community you have to use another door. Wait a minute, if a door is a network interface, does it mean that the gatekeeper house has another door? Absolutely correct! The house 11 has another door with another number on it (101) and of course this door has another serial number (MAC Address).

By exiting this door you can now take the path to reach community 20th which has its own gatekeeper in house 21. The map (Routing Table) of this gatekeeper directs you to the proper door to reach your destination. This door gives you access to community 20th as your destination 24 belongs to it. The gatekeeper also give you the map (ARP table) so you can orientate yourself at the fountain. You can now walk on the path to the green fountain. From there, you just have to follow the sign and the path to house 24. When you want to get back home, you travel back using the same path in the opposite direction.

Networking in Kubernetes

Now you know the basics about networking, let’s see how it works in comparison in Kubernetes. Yes it is slightly more complicated but let’s go step by step and use the picture below to understand it better:

Instead of houses, we now have buildings. The networking between the buildings is still the same as the traditional one with a Switch/fountain in the middle. The entrance of the building has a door with the building number on it (its IP Address) that is part of a 1000th community. One building will represent one node of our Kubernetes cluster.

You know Kubernetes is a container orchestrator. A container is wrapped into a pod. For simplicity sake, let’s consider a pod has only one container and so both terms are here equivalent. This pod will be one appartement in our building and so is like a private part of it. The apartment could be of different size in the building as it could have 2, 3 or 4 bedrooms for example. This will be the CPU and memory capacities your container will need on the node. Some appartement are empty so the building still has some capacity. However in Kubernetes, pods are created and deleted as needed. So in our building that would mean sometimes a 2 bedrooms appartement is created and when not used anymore, it could be removed from the building. Then a 5 bedrooms appartement could be created next if the building has enough space for it. Imagine then that it is a LEGO building and inside it you can build and demolish apartments of various size according to your need! Isn’t it awesome?

In each building, the containers/pods have their own community (IP subnet). The function of a CNI in Kubernetes is basically to assign numbers (IP Addresses) to pods so they can communicate together. By default Cilium uses different community for each building. When an apartment is created, Cilium assigns a number to it. When an apartment is deleted an recreated, it will get another number so it is ephemeral. Here the blue community uses the 10th range and the green one uses the 20th. You can notice that the number ranges of the blue and green communities are different than the range of the buildings. Just for you to know, this design is called Overlay network, there are other ones possible but that is a common one. There is then a network of pods on top of the network of the nodes.

Traveling between apartments in the same building

Now, you live in the apartment 12, how are you going to pay a visit to apartment 14? Like we did with the traditional networking example, you are the packet we are going to follow! Of course you leave the apartment (container) through its door (its network interface). The difference with our previous example is that you are now not out of the house, you are just out of your apartment but still inside the building. You then follow a private corridor and reach another door (this is the LXC interface).

This door gives you access to a common space of the building where the routing and dispatching occurs. We call it the Cilium Lobby (the blue rectangle). This Lobby has been installed by the Cilium Agent of this building when Cilium was selected to provide communications in this Kubernetes cluster. There is one Cilium Agent per building and he is taking care of everything related to networking.

So you reach the fountain in that Lobby (it is a virtual Switch here) with all its signs. You have in your pocket the map (ARP table) to decipher them and find a match with number 14. It is the door on the top right. You then open that door, follow the corridor and reach apartment number 14. You would go back to apartment number 12 following the same path but in the opposite direction.

As the destination is in the same community, it is still Layer 2 routing and you can see the similarities with traditional networking.

Traveling between apartments in different buildings

Now from apartment 12 you want to pay a visit to the apartment number 22 which is in another building. The beginning of your travel is the same as previously, you exit your apartment, follow the corridor, reach the Cilium Lobby and the fountain. As with traditional networking, the destination number belongs to another community so you need the help of the gatekeeper in the Lobby. The difference is that this gatekeeper doesn’t live in an apartment, he is just waiting at a deck in the Lobby. The gatekeeper looks at his map (the Routing table) for direction to number 22 and shows you the door number 11 to use (the cilium_host).

When you open that door, you see another door behind it: It is the blue triangle and it is called the VXLAN interface. This door open to a nice transparent tunnel that goes through the main door of the building. You are protected from the rain and can enjoy the landscape while walking to the other building. You even spot the outdoor fountain! When you reach the green building you exit the tunnel, greet the cilium_host number 21 of this building, reach the Lobby and go to the gatekeeper desk. You gave him your destination (apartment 22) and he shows you the path to the fountain of this building. He also gave you the map (ARP Table) so you can translate the signs at the fountain. From the fountain you head now to the door on the top left, follow the corridor and reach your destination. As before, your way back will follow the same journey in the opposite direction.

This is Layer 3 routing as the destination community is different than yours. Although the routing is a bit more complex in Kubernetes, the principle stays the same as with traditional routing: Finding the right door to reach your destination number. Note that there are other modes of routing but this one is the a common one.

Wrap up

I hope this helped you understand the difference between the traditional networking and Kubernetes networking and also that the latter is clearer now for you. If that is all you needed, then I’m glad you’ve read this blog post, I hope you’ve enjoyed it. If you now want to learn more about Kubernetes Networking, stay tuned because I’ll write an intermediate level where you’ll see what a building really looks like on a real cluster!

L’article Kubernetes Networking by Using Cilium – Beginner Level est apparu en premier sur dbi Blog.

M-Files January Release 2024

Yann Neuhaus - Tue, 2024-02-13 12:15

A bunch of new features and a seamless user experience across all the M-Files products have been introduced with the M-Files January 2024 Release. In addition, it will bring a fresh look across all M-Files products and additional Business Benefits.
In this blog, I will give a summary of the new Features and Business Benefits including pictures of the new look and feel of the different M-Files products.
Furthermore, I will also share the instruction to follow, in order to keep the former colour scheme.

Business Benefits of the M-Files January Release 2024

Outlined below you will find a list of the most important Business Benefits.

  • Consistent User Experience: Streamlined and cohesive design across products for a consistent and intuitive user experience.
  • Enhanced Accessibility: Improved color contrast and accessibility features will cater to a broader audience, ensuring a positive experience for users with diverse needs.
  • Efficient Learning Curve: Quicker adaptation to the unified design, as common elements and patterns create a more efficient learning curve.
  • Increased User Satisfaction: A cohesive and accessible design contributes to higher user satisfaction.
Summary of the main new Features of the M-Files January Release 2024

Features below will give you a list of the major improvements.

  • M-Files Desktop Client and M-Files Web Client have the same visual identity.
  • In the M-Files comments feature you have the option to mention someone by using the @ plus the name. In addition, this will trigger a notification.
  • M-Files Desktop is supporting the copy of many links at once. You can now select many objects and create links to them at once in M-Files Desktop.
  • It is now possible to set the session timeout individually. This can be easily done under the Advanced Vault Settings section.
  • To streamline the process, improved sharing options were implemented.
    Description out of the M-Files Release Notes: “Sharing options in M-Files Desktop have been clarified, and rarely used features have been removed if M-Files Web is enabled. PDF options have been moved to a “Save as PDF” section of the context menu. These are user interface changes only and have no effect on the API.“.

Joonas Linkola (Senior Partner sales Engineer at M-Files) shared a nice summary of the release note on LinkedIn.

For further details, please take a look at the official M-Files Release Note.”

How it looks now… Reverting to the previous M-Files colour scheme

Open the M-Files Admin application and navigate to the Vault you would like to modify.
Click on Configuration => Client => Desktop => Appearance.
Particularly, the parameters below have to be adapted.

  • Enabled > Yes
  • Logo Image > M-Files Logo below
  • Top Area Color > #318ccc
  • Accent Color > #318ccc
  • Top Area Breadcrumb and Icon Color > #ffffff

Please feel free to look at the below video that shows the same, step by step.

L’article M-Files January Release 2024 est apparu en premier sur dbi Blog.

Cloud Native Storage: Overview

Yann Neuhaus - Tue, 2024-02-13 09:21

When thinking of containers, Cloud Native Storage topic shines like an evidence. You have to handle this topic with care as it will carry your workload and configuration. This blog will be the first of a series talking about CNS and usage.
I will try to cover as many aspects as possible and clear up the CNCF situation when you start looking at the landscape. The purpose of this series will not be to a direct comparison of the products one to one. Especially because of the number of products and time consuming testing. Plus you may find many on the internet who did it for different purposes and different products. It will be more a higher level method to choose wisely what will the best fit your needs and workload.

Before I go further, let me just raise a little warning about what you’ll read below. To understand terminology and the ins and outs concepts, you better be familiar with Kubernetes. For example, holding a CKA (Certified Kubernetes Administrator) may help you better appreciate this series of blogs.

Classic storage, I mean at the OS level, is what we call ephemeral storage from a container view. You may have LVM, nfs shares or any storage managed at the OS level. Many SRE engineers will stay at this level for data protection to secure and avoid the risk of loosing their precious data. For example, we may imagine a database administrator having his database installed in a VM and thinking to backup the dedicated file system where the data are stored.

Now let’s get back to cloud native storage, as the name depict it, it has been built specifically for cloud environments. Stricto sensu, cloud differs from on-premise for hosting, but here, we’re describing the general term for “cloud native”. It means a software approach to develop, build, create and manage modern applications and for that, they require scalable, flexible and resilient capabilities. These applications can leverage on public, private or hybrid cloud infrastructures.

What are these cloud native applications, they are (or should be) most of the time small and independent services allowing a very fast release cycle with more confidence. This can also be part of the efficient time-to-market definition. Again, small doesn’t mean it can’t handle very big loads, scalability and flexibility are part of the cloud native’s main purpose. In fact, it is more designed for efficiency and reliability to handle production environments by reducing cost in a general manner when load is varying.

You may also find big monolithic workloads in containers and I can say that, since I work with containers, I saw a lot of productive environments running applications that are not microservices. It is sometimes not possible due to vendor products or it can be a vendor first move before moving from monolithic to microservices.

Now, let’s talk about the CNCF landscape regarding Cloud Native Storage, it is a sub-topic in the Runtime category.
You can find it by following the link.

If you are not familiar with this view, let me give you some hints

  • Some product have a grey background, it’s because their code is proprietary
  • Some product have a white background, it’s because their code is open-source
  • Projects are graduated regarding vote from the community, it indicate the maturity level of a product
  • Graduation can be the following
    • Sandbox: Innovators
    • Incubating: Early adopters
    • Graduated: Majority
    • Archived: Grey logo, Project no longer supported
  • Some are in this picture only because they support the community as members
    • Platinum
    • Gold
    • Silver
    • Academic
    • Nonprofit
    • End User supporter

I can imagine now what can be your next question is: Which product should I use for my needs.
Well, this will be the topic of my next blog series regarding Cloud Native Storage.

In the mean time, I encourage you to follow my colleague’s blog on not being afraid to install a database in Openshift here.

L’article Cloud Native Storage: Overview est apparu en premier sur dbi Blog.

Configure Zabbix with Ansible

Yann Neuhaus - Tue, 2024-02-13 07:14

Besides the ability to setup your Zabbix server, proxy and agents, the Ansible collection for Zabbix (ie. community.zabbix) also allows to configure every aspects of your Zabbix system. From hosts to users, discovery rules or even templates. In this blog post, we will see how this module helped me to automatically configure hosts I provision with YaK in my Zabbix server configuration.

Setup community.zabbix

The first step is easy and fast. In your YaK environment, run the following command:

ansible-galaxy collection install community.zabbix

This collection is actually an interface with Zabbix API which make our work easier when we already know Ansible and don’t want to go through complex HTTP API calls (see my other blog post about it).

Setup Ansible Variables

If you know Ansible, you know that all tasks are going through a ssh connection to the target host. Once agent is installed on target host, we want to configure it in Zabbix server. This is achieved by modifying few variables before calling zabbix modules:

- name: Set api configuration facts
  ansible.builtin.set_fact:
    ansible_network_os: community.zabbix.zabbix
    ansible_connection: httpapi
    ansible_httpapi_port: 80
    ansible_zabbix_url_path: ''
    ansible_host: "{{ zabbix_server_ip }}" # Zabbix server
    ansible_user: "{{ zabbix_api_login_user }}"

For requests to be redirected, all tasks concerning our host must go to the zabbix server (line 7). Zabbix server does not understand ssh connection, but httpapi connections (line 4) with ansible_network_os specific to the API (line 3). Authentication against the http api is made with ansible_user as login and ansible_httpapi_pass as password. For better security, we could create an API token and use https.

Usage Host Group

Before creating our host, we can start with a simpler task like creating a host group:

- name: Create host groups
  community.zabbix.zabbix_group:
    host_group: "{{ zabbix_agent_groups }}"
  register: zabbix_api_hostgroup_created
  until: zabbix_api_hostgroup_created is succeeded

This task calls zabbix_group module with a list of group in host_group argument. Could it be more easier than that?

Note that lines 4 and 5 are there to ensure host group is created as concurrent access (while running against multiple hosts) might raise creation failure. The default value for retries is 3 and with a delay of 5 seconds between each retry.

Host

To create a host in Zabbix configuration, there are a bit more variables to provide to the module:

- name: Create host in zabbix server configuration
  community.zabbix.zabbix_host:
    host_name: "{{ hostname }}"
    host_groups: "{{ zabbix_agent_groups }}"
    link_templates: "{{ zabbix_agent_link_templates }}"
    interfaces: "{{ zabbix_agent_interfaces }}"
    tags: "{{ zabbix_agent_tags }}"
    tls_connect: 2 # PSK
    tls_psk_identity: "{{ zabbix_agent2_tlspskidentity }}"
    tls_psk: "{{ zabbix_agent2_tlspsk_secret }}"
  register: zabbix_api_host_created
  until: zabbix_api_host_created is succeeded

host_name is the how the host will be named in Zabbix configuration (coming from inventory – line 3). host_groups is the list of groups we’ve created previously. link_templates is a list of Zabbix templates.

interfaces (line 6) will be the list of defined interfaces to reach the Zabbix agent from the Zabbix server. This variable content looks like:

    zabbix_agent_interfaces:
      - type: 1
        main: 1
        useip: "{{ zabbix_useuip }}"
        ip: "{{ ansible_host }}"
        dns: ""
        port: "{{ zabbix_agent_listenport }}"

Next tls_* variables are set to encrypt trafic between agent and server.

tags variable allows to have host added with tags in Zabbix configuration.

Et voilà!

L’article Configure Zabbix with Ansible est apparu en premier sur dbi Blog.

Adding a PDB (and Service) to a RAC database -- 1 - service running on only 1 Instance

Hemant K Chitale - Tue, 2024-02-13 02:10
On my existing, RAC database I have :

[oracle@node1 ~]$ srvctl status database -db DB21CRAC
Instance DB21CRAC1 is running on node node1
Instance DB21CRAC2 is running on node node2
[oracle@node1 ~]$
[oracle@node1 ~]$ srvctl status service -d DB21CRAC -s hemantpdb
PRKO-2017 : Service hemantpdb does not exist for database DB21CRAC.
[oracle@node1 ~]$ srvctl status service -db DB21CRAC -service hemantpdb -pdb HEMANTPDB
PRKO-2017 : Service hemantpdb does not exist for database DB21CRAC.
[oracle@node1 ~]$
[grid@node1 ~]$ lsnrctl status LISTENER_SCAN1

LSNRCTL for Linux: Version 21.0.0.0.0 - Production on 13-FEB-2024 15:06:11

Copyright (c) 1991, 2021, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER_SCAN1
Version                   TNSLSNR for Linux: Version 21.0.0.0.0 - Production
Start Date                13-FEB-2024 14:51:13
Uptime                    0 days 0 hr. 14 min. 57 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u01/app/21.3.0.0/grid/network/admin/listener.ora
Listener Log File         /u01/app/grid/diag/tnslsnr/node1/listener_scan1/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN1)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.56.115)(PORT=1521)))
Services Summary...
Service "0f488ad896262f80e0636f38a8c0fc18" has 2 instance(s).
  Instance "DB21CRAC1", status READY, has 1 handler(s) for this service...
  Instance "DB21CRAC2", status READY, has 1 handler(s) for this service...
Service "DB21CRAC" has 2 instance(s).
  Instance "DB21CRAC1", status READY, has 1 handler(s) for this service...
  Instance "DB21CRAC2", status READY, has 1 handler(s) for this service...
Service "DB21CRACXDB" has 2 instance(s).
  Instance "DB21CRAC1", status READY, has 1 handler(s) for this service...
  Instance "DB21CRAC2", status READY, has 1 handler(s) for this service...
Service "hemantpdb" has 2 instance(s).
  Instance "DB21CRAC1", status READY, has 1 handler(s) for this service...
  Instance "DB21CRAC2", status READY, has 1 handler(s) for this service...
The command completed successfully
[grid@node1 ~]$
[oracle@node1 ~]$SQL> select inst_id, name, network_name from gv$services order by 1;

   INST_ID NAME             NETWORK_NAME
---------- ---------------- ----------------
         1 hemantpdb        hemantpdb
         1 DB21CRAC         DB21CRAC
         1 SYS$BACKGROUND
         1 DB21CRACXDB      DB21CRACXDB
         1 SYS$USERS
         2 hemantpdb        hemantpdb
         2 DB21CRAC         DB21CRAC
         2 SYS$BACKGROUND
         2 DB21CRACXDB      DB21CRACXDB
         2 SYS$USERS

10 rows selected.

SQL>
SQL> select inst_id, con_id, name, open_mode from gv$pdbs order by 1,2;

   INST_ID     CON_ID NAME             OPEN_MODE
---------- ---------- ---------------- ----------
         1          2 PDB$SEED         READ ONLY
         1          3 HEMANTPDB        READ WRITE
         2          2 PDB$SEED         READ ONLY
         2          3 HEMANTPDB        READ WRITE

SQL>


Which means that I have created a custom PDB called "HEMANTPDB" and there is a default service called "hemantpdb" on each instance.  However, this services is NOT listed when I check via srvctl. 

This is because srvctl queries the cluster for information about services.  Service"hemantpdb" is created by default when I run CREATE PLUGGABLE DATABASE HEMANTPDB.


So, I can add a new service and configure Transparent Application Failover for SELECT failover but with only the first instance(DB21CRAC1) as the only one to start the service on initially{and the second instance (DB21CRAC2) as the alternate} :

[oracle@node1 ~]$ srvctl add service -db DB21CRAC  -service newservice -preferred DB21CRAC1 -available DB21CRAC2 -tafpolicy BASIC -failovertype SELECT -pdb HEMANTPDB
[oracle@node1 ~]$
[oracle@node1 ~]$ srvctl config service -db DB21CRAC  -service newservice
Service name: newservice
Server pool:
Cardinality: 1
Service role: PRIMARY
Management policy: AUTOMATIC
DTP transaction: false
AQ HA notifications: false
Global: false
Commit Outcome: false
Reset State: NONE
Failover type: SELECT
Failover method:
Failover retries:
Failover delay:
Failover restore: NONE
Connection Load Balancing Goal: LONG
Runtime Load Balancing Goal: NONE
TAF policy specification: BASIC
Edition:
Pluggable database name: HEMANTPDB
Hub service:
Maximum lag time: ANY
SQL Translation Profile:
Retention: 86400 seconds
Failback :  no
Replay Initiation Time: 300 seconds
Drain timeout:
Stop option:
Session State Consistency: DYNAMIC
GSM Flags: 0
Service is enabled
Preferred instances: DB21CRAC1
Available instances: DB21CRAC2
CSS critical: no
[oracle@node1 ~]$
[oracle@node1 ~]$ srvctl start service -db DB21CRAC -service newservice
[oracle@node1 ~]$ srvctl status service -db DB21CRAC -service newservice
Service newservice is running on instance(s) DB21CRAC1
[oracle@node1 ~]$




If the database instance DB21CRAC1 fails (crashes), the ClusterWare starts the service on DB21CRAC2.  Here I kill the DB21CRAC process and then verify that the service has restarted on DB1CRAC2  :

[oracle@node1 ~]$ srvctl status service -db DB21CRAC -service newservice
Service newservice is running on instance(s) DB21CRAC1
[oracle@node1 ~]$ ps -ef |grep smon
oracle    2951     1  0 16:05 ?        00:00:00 ora_smon_DB21CRAC1
root      3521     1  1 14:50 ?        00:00:48 /u01/app/21.3.0.0/grid/bin/osysmond.bin
grid      4068     1  0 14:50 ?        00:00:00 asm_smon_+ASM1
oracle    4146 25526  0 16:05 pts/0    00:00:00 grep --color=auto smon
[oracle@node1 ~]$ kill -9 2951
[oracle@node1 ~]$ srvctl status service -db DB21CRAC -service newservice
Service newservice is running on instance(s) DB21CRAC2
[oracle@node1 ~]$


In this case, connections using this service name ("newservice") will connect (failover) to the 2nd database instance running on node2 of the Cluster.

Categories: DBA Blogs

Happy 21st Birthday to PeteFinnigan.com Limited

Pete Finnigan - Mon, 2024-02-12 18:06
My company PeteFinnigan.com Limited is 21 years old today!! It seems that time has gone so fast. When I started the company my oldest son was a baby and now he is almost 22 years old and works here in....[Read More]

Posted by Pete On 12/02/24 At 11:28 AM

Categories: Security Blogs

SQL loader not loading all the needed rows due to new line character and enclosement character

Tom Kyte - Mon, 2024-02-12 18:06
I have a problem with how SQL loader manage the end of a column value. I was hoping to manage CR LF, the enclosement character and the separator character but it seems I can't find a solution! The data I receive from the .csv file looks like this: <code>"C","I","FLAGS","LASTUPDATEDATE","BOEVERSION","C_OSUSER_UPDATEDBY","I_OSUSER_UPDATEDBY","C_OSUSER_PWF","DESCRIPTION","DURATION","ENDDATE","I_OSUSER_PWF","LASTSTATUSCHA","STARTDATE","DURATIONUNIT","TYPE","STATUS","C_BNFTRGHT_CONDITIONS","I_BNFTRGHT_CONDITIONS","C_CNTRCT1_CONDITION","I_CNTRCT1_CONDITION","EXTBLOCKTYPE","EXTBLOCKDURATIONUNIT","EXTBLOCKDURATION","EXTBLOCKDESCRIPTION","PARTITIONID" "7680","423","PE","2015-07-06 11:42:10","0","1000","1506","","No benefits are payable for a Total Disability period during a Parental or Family-Related Leave, for a Total Disability occurring during this period. ","0","","","","","69280000","69312015","71328000","7285","402","","","","","","","1" "7680","426","PE","2015-07-06 11:42:10","0","1000","1506","","""Means to be admitted to a Hospital as an in-patient for more than 18 consecutive hours. "" ","0","","","","","69280000","69312021","71328000","7285","402","","","","","","","1"</code> My ctl file is as follows: <code>Load Data infile 'C:\2020-07-29-03-04-48-TolCondition.csv' CONTINUEIF LAST != '"' into table TolCondition REPLACE FIELDS TERMINATED BY "," ENCLOSED by '"' ( C, I, FLAGS, LASTUPDATEDATE DATE "YYYY-MM-DD HH24:MI:SS", BOEVERSION, C_OSUSER_UPDATEDBY, I_OSUSER_UPDATEDBY, C_OSUSER_PWF, DESCRIPTION CHAR(1000), DURATION, ENDDATE DATE "YYYY-MM-DD HH24:MI:SS", I_OSUSER_PWF, LASTSTATUSCHA DATE "YYYY-MM-DD HH24:MI:SS", STARTDATE DATE "YYYY-MM-DD HH24:MI:SS", DURATIONUNIT, TYPE, STATUS, C_BNFTRGHT_CONDITIONS, I_BNFTRGHT_CONDITIONS, C_CNTRCT1_CONDITION, I_CNTRCT1_CONDITION, EXTBLOCKTYPE, EXTBLOCKDURATIONUNIT, EXTBLOCKDURATION, EXTBLOCKDESCRIPTION, PARTITIONID)</code> Here is what I tried in the control file: CONTINUEIF LAST != '"' CONTINUEIF THIS PRESERVE (1:2) != '",' "str X'220D0A'" Here is the result I currently have with "CONTINUEIF LAST != '"' <code>Record 2: Rejected - Error on table FNA_FNTFO2.TOLCONDITION, column DESCRIPTION. second enclosure string not present Record 3: Rejected - Error on table FNA_FNTFO2.TOLCONDITION, column C. no terminator found after TERMINATED and ENCLOSED field Table FNA_FNTFO2.TOLCONDITION: 1 Row successfully loaded. 2 Rows not loaded due to data errors. 0 Rows not loaded because all WHEN clauses were failed. 0 Rows not loaded because all fields were null.</code> Is there any way to manage line break and enclosement character in SQL Loader? I dont understand why we can`t change how it sees rows. Instead of seeing a new row when there is a CR LF, can we tell it to concacenate values until the last enclosement character (chr34 in my case) + the separator character (y, in my case) has been seen. I really ho...
Categories: DBA Blogs

Reestablish administrator role access to Rancher users

Yann Neuhaus - Mon, 2024-02-12 07:49
Introduction

I came across a case with no more Administrator users in Rancher. Therefore there was no access to the local (Rancher) cluster and administration of Rancher. No user could configure and reassign the Administrator role. I will show you how to retrieve the Administrator role for your Rancher users.

Solution

The current default admin user has the role Restricted-Administrator. He is no longer an Administrator and cannot reassign himself to the Administrator role due to insufficient rights.

error message when assigining Administrator role

Let’s create a new user called temporary-admin with User-Base permission.

Get the ID of the user and access your Rancher Kubernetes cluster. Use a config file that can modify ClusterRoleBinding in the cluster.

Search for the ClusterRoleBinding associated with the user ID, here u-7s8dx.

$ kubectl get clusterrole | grep 7s8
u-7s8dx-view                                                           2024-02-07T15:38:53Z
$ kubectl get clusterrole u-7s8dx-view -o yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  creationTimestamp: "2024-02-07T16:16:05Z"
  finalizers:
  - wrangler.cattle.io/auth-prov-v2-crole
  labels:
    cattle.io/creator: norman
  name: u-7s8dx-view
  ownerReferences:
  - apiVersion: management.cattle.io/v3
    kind: User
    name: u-7s8dx
    uid: f409c575-e413-429d-8860-ca2e8103e736
  resourceVersion: "5803466"
  uid: 23d7347c-838c-4ed4-a1c2-290c782384ad
rules:
- apiGroups:
  - management.cattle.io
  resourceNames:
  - u-7s8dx
  resources:
  - users
  verbs:
  - get

Edit the ClusterRole with the following rules:

rules:
- apiGroups:
  - '*'
  resources:
  - '*'
  verbs:
  - '*'
- nonResourceURLs:
  - '*'
  verbs:
  - '*'

Now the user temporary-admin should have the permissions to reassign the Administrator role to your user. Edit your user so they can be Administrator again, log into it, then delete the temporary-admin user.

Now, your user has the Administrator role back in Rancher, and all your local modifications with the temporary-admin user have been deleted.

Check our other blog articles

https://www.dbi-services.com/blog/category/devops/
https://www.dbi-services.com/blog/category/kubernetes/

L’article Reestablish administrator role access to Rancher users est apparu en premier sur dbi Blog.

Object_id

Jonathan Lewis - Sun, 2024-02-11 12:23

This is a note that will probably be of no practical use to anyone – but someone presented me with the question so I thought I’d publish the answer I gave:

Do you know how object ID is assigned? It doesn’t look as if a sequence is used

I’m fairly sure the mechanism has changed over versions. (Including an enhancement in 23c where the object number of a dropped (and purged) object can be reused.)

I don’t really know what Oracle does, but I do know that there is an object in obj$ called “_NEXT_OBJECT” and Oracle uses that as a control mechanism for the dataobj# and obj# (data_object_id and object_id) in some way. I think Oracle uses the row a bit like the row used by a sequence in seq$ – the dataobj# is bumped by a small value (seems to be 5) each time it is reached (cp. seq$.highwater for a sequence) and it’s possible that the obj# is used to record the instance number of the instance that bumped it. The instance then (I think) has a small cache of obj# values it can use before it has to read and update the “_NEXT_OBJECT” row again.

Footnote.

You might note that this description means that it is the dataobj# that actually drives the generation of a new obj# / object_id. You can demonstrate this most easily (if you have sole access to the database) by:

  • creating a table,
  • checking its object_id and data_object_id (which will match),
  • moving it a couple of time (which will increment the data_object_id – and only the data_object_id – each time),
  • creating another table.

The second table will have an object_id that is one more than the current data_object_id of the first table.

The Beginning Of The End

Michael Dinh - Sat, 2024-02-10 20:05

Started blogging June 19, 2007.

Thank you for reading!

No better way to end it than the year of Dragon [Wood] (Chinese Horoscope).

Be well; be kind; be happy.

Playlist of Oracle RAC Videos

Hemant K Chitale - Sat, 2024-02-10 02:23

 This is a link to my YouTube playlist of demonstrations on Oracle RAC (12c and 21c)



Categories: DBA Blogs

Generate java code in SQL/PLUS

Tom Kyte - Thu, 2024-02-08 22:26
Hi, Tom, How are you. I have below source code: create or replace and compile java source named "Something" as import oracle.sql.*; public class Something { ............... } When i wrote above java source file under SQLPLUS, I got the following errors: ERROR at line 1: ORA-29536: badly formed source: Encountered "<EOF>" at line 1, column 20. Was expecting: ";" ... It complains my ";" after "import oracle.sql.*", so do i need grant some priveleges or others? Thanks ============================ Hi Tom, My database is Oracle 8i, the message under SQLPLUS is: SQL*Plus: Release 8.0.6.0.0 - Production on Fri Jun 22 10:12:30 2001 (c) Copyright 1999 Oracle Corporation. All rights reserved. Connected to: Oracle8i Enterprise Edition Release 8.1.6.1.0 - Production With the Partitioning option JServer Release 8.1.6.0.0 - Production But when i executed the following, i got: dummy@someserver> create or replace and compile java source named 2 "Somthing" 3 as 4 import oracle.sql.*; "Somthing" * ERROR at line 2: ORA-29536: badly formed source: Encountered "<EOF>" at line 1, column 20. Was expecting: ";" ... How should i do? Thanks
Categories: DBA Blogs

Pages

Subscribe to Oracle FAQ aggregator