Friday, February 10, 2017

TDI / SDI - Connect to Active Directory over SSL - How to

I mostly write this post as a reminder to my self the next time I have a similar need.

I had a case where I had to connect to Active Directory to be able to create users and set passwords on that user, off course, using TDI.

The AD administrator gave me a .pfx format of the certificate which is stored in AD.
Installing this file in Windows is easy. Just double click it and install.
Then, starting the "certmgr.msc" from Start - Run inn Windows, I was able to right click the cert, selected "export".

Then, go like this:
And select the DER format:

Saved this exported cert on d:\temp as "cert.der"

Then, open up a command prompt and go to the tdi\jvm\jre\bin catalog:

cd D:\IBM\TDI\V7.1.1\jvm\jre\bin

Then, create a .jks keystore and import the cert.der into it:

keytool -import -file d:\temp\cert.cer -keystore ADKEYSTORE.jks -storepass PaSsW0Rd -alias ADKEYSTORE

If all goes good, output will be:

Issuer: CN=FS03-CA, DC=CUSTOMER, DC=local
Serial number: 7a638e0a000000000001
Valid from: 10.02.17 14:20 until: 10.02.19 14:30
Certificate fingerprints:
MD5:  F8:2E:4B:C7:1B:04:58:5F:E1:FF:2E:B1:88:EE:02:4A
SHA1: 06:97:8F:E":93:21:FB:BB:71:E2:C2:FF:02:06:17:8E:8E:02:8C:A5
Trust this certificate? [no]:  yes
Certificate was added to keystore

And to check the content of the .jks keystore:
keytool -list -keystore ADKEYSTORE.jks -storepass PaSsW0Rd

Output will be:

Keystore type: jks
Keystore provider: IBMJCE

Your keystore contains 1 entry

ADKEYSTORE, 10.feb.2017, trustedCertEntry,
Certificate fingerprint (MD5): F8:2E:4B:B7:1B:14:58:5F:A1:FF:2E:91:88:3E:02:4A

I then moved the ADKEYSTORE.jks file to my TDI Solutions directory, which is in this case:

over to:

I then modified the file "E:\TDISOL\TDI_custom\"

Where I inserted:

#server authentication
#client authentication

After restarting TDI, I was able to connect to the AD server on port 636 in TDI.

Thursday, January 19, 2017

IBM Connections 5.5 - Make top header/menu sticky

Update: Now also supports Safari on Mac. I got a comment on this blog-entry from Stanislav Shvachko, who pointed out that it does not work on Safari on Mac.
I went through the code and fixed it. So no it also works for Safari, Internet Explorer 11, Firefox and Chrome.

Original Blog:

I had this challenge given to me from a customer;
"Can you make the top menu/header always visible when scrolling the pages downwards in Connections?"

Originally, my answer was NO, because this is not an out-of-the-box option.

But, being the html/css enthusiast as I am, I started looking into it.

I played around a bit with it in Firefox using Firebug. And I noticed that when I put in a "position: fixed" css property on the correct Divs, I was actually getting somewhere.

Along the way, I tested all the different apps in Connections, and in some of the apps, like Homepage, Files and the Profiles Directory Search, there were already some elements that was "sticky" when you were scrolling down.

Plus, when I tried accessing the top-menus in Connections, after I put the header in sticky-mode, the drop-down menus needed some tweaking regarding it´s position.

So, in the end, here´s the custom.css that I came up with.

I can not promise that I´ve covered all the areas and that this will work in all browsers, in all scenarios. But this one worked in my Connections 5.5 CR2 server in Firefox, Internet Explorer 11 and in Chrome.

NOTE that if you have a logo already present in the custom.css, as described here you need to edit my css regarding the "top" and the "margin-top" pixel sizes. Because if you have a logo which is larger in height and is pushing the header-height downwards a bit, you need to add more pixels in those 2 css properties.

Ok, if you don´t have a "custom.css" already in place in you Connections installation, create this file in the "shared\customization\themes\hikariTheme" directory. (You might have to create this directory structure if it´s not already there)

Then, using your favorite text-editor, paste this into it:

/* Header code */
.lotusui30 .lotusBanner {
    background: #325c80 none repeat scroll 0 0;
    padding: 0;
    position: relative;
    width: 100%;
    z-index: 11;
    overflow: visible;
    text-align: left;

/* The top header/banner/menu - added position and width. The rest is default values. */
.lotusui30 div.lotusBanner .lotusRightCorner {
    background: #325c80 none repeat scroll 0 0;
    height: 44px;
    overflow: hidden;
    padding-bottom: 1px;
    position: fixed; /*NEW*/
    width: 100%; /*NEW*/

/* The body below the header/banner/menu, needs top padding of 42px */
.lotusui30 .lotusTitleBar, .lotusui30 .lotusTitleBar2 {
    background: rgba(0, 0, 0, 0) none repeat scroll 0 0;
    border: 0 none;
    border-radius: 0;
    margin: 0;
    padding: 42px 0 0; /*NEW*/

/* Fix for Files app - lefmenu which is already sticky. Adding some top margin */
.files-independent-scrollbars-compatible .lotusui30_body .lotusMain {
    padding-left: 50px;
    padding-right: 0;
    top: 42px;

/* Fix for the Homepage "what do you want to share" box, which was already sticky. Added som top margin */
.lotusStream #activityStreamMain.lotusWidgetBody .streamHeaderWrapper.isSticky {
    margin-top: 42px; /*NEW*/
    position: fixed; /*NEW*/
    top: 0;
    z-index: 2;

/* Fix for the homepage left-hand side menu, which was already sticky. Added some top margin */
#homepageLeftNavigationMenuContainer.isSticky {
    margin-top: 42px !important; /*NEW*/
    position: fixed; /*NEW*/
    top: 10px;
    width: 215px;

/* Fix for the dropdown menus in the banner. Needed 42px top margin */
.dijitPopup {
    background-color: transparent;
    border: 0 none;
/*    margin: 42px 0 0; /*NEW*/ */
    padding: 0;
    position: absolute;

/* Fix for the sticky grey search bar in Directory Search. Needed 45px top */
.lotusui30 .lconn_directoryPage .lconn_searchNode.fixed {
    background-color: #f0f0f0;
    padding-top: 15px;
    position: fixed; /*NEW*/
    top: 45px !important; /*NEW*/
    z-index: 900;

/* Thanks to @robertfarstad I now have a sticky top banner in Connections */

And then, a simple restart of the "Common" application should suffice, in order for you to see the changes. (Perhaps a browser cache wipe as well).

If you have issues with the fact that custom.css is not being picked up at all by Connections, you might find some tips here:

I ask of you. If you try this, could you please notify me if there´s anything that I´ve missed regarding sticky stuff? Positioning of other widgets, menus and such is something that can be tuned in this css, but I need to know which page/app you see that it´s not working on.
If you have a test server, it´s real simple to test this out. So please do and let me know the result, good or bad :-)

I have some fellow IBM Connections friends on Skype that is going to test this. I will update this blog post with their findings and results as well.

Oh, yeah... Here´s the result on "my profile" page:

Wednesday, January 18, 2017

IBM Connections 5.5 - change the name of "My Organization Communities"

I noticed a new feature In Connections 5.5.
The menu item for "Public Communities" has changed to "My Organization Communities". This is something we´ve all seen.

In CNX5.0: In CNX 5.5:

What I didn´t know, was that there´s now a possibility to change the name of "My Organization" to be of your choosing. There´s a new configuration in LotusConnections-config.xml called "<organization name="">

This you can change to <organization name="Your Company Name">

The recipe to do this is here:

Restrictions are as follows:
  1. The company name can only be maximum 20 characters long
  2. For special characters, like ÆØÅ in Norwegian, you have to use Unicode.
 That last one applies to all languages that has special characters. Norwegian, German, Swedish.... etc.

Format is like this:
For the company name "Øre Nese Hals", you have to change to this, in LotusConnections-config.xml:
<organization name="\u00d8re Nese Hals">
 Where the capital letter "Ø" equals "\u00d8"

Or, by doing it via wsadmin:
LCConfigService.updateConfig("","\u00d8re Nese Hals")

So, for my company name, which has no special characters, this is what I got:

Monday, December 19, 2016

IBM Connections - Force setting the "share with external" option on files.

Open for debate, please comment if you have any insight in why this might not be a good thing to do.

When you upload a file in IBM Connections Cloud, the option "Files can be shared with people external to my organization" is automatically set for you:

Which is a good thing.

In On-Prem versions of IBM Connections (I tested this in Connections 5.5 CR2.), this option is left unchecked, so in order for you to be able to share this file with external users, you have to remember to set this EVERY TIME you upload a file.
Because, if you forget to set it, there is no way of setting this after the file has been uploaded.

Which is a bad thing.

And the only place this option exists, is when you use IBM Connections in a Browser!!
If you upload files in the Windows Desktop Plugin, using the Windows Explorer addition, this option does not even exist!!
See for yourself:

I think the whole "share with external users" is poorly made in IBM Connections. This could have been programmed a whole lot better.

Just the fact that if you forget to select the "Files can be shared with people external to my organization" option, you are not able to set it afterwards, is just incredible....

But, opinions aside, I started looking at possibilities to set this "share with external" option automatically, and so far, this is what I came up with:

First and foremost, you have to set your user, and the other users who should be able to share with external users, to actually be able to do just that. (This is for On-Prem only, of course)

This includes the running wsadmin command
ProfilesService.setRole("", EMPLOYEE_EXTENDED)

(See more here:

So, as we all know, there is a FILES database.
In this database, there is a table called "MEDIA".
This table has a list of all the files that´s been uploaded into IBM Connections.

So I did a test: I uploaded a file, leaving the "Files can be shared with people external to my organization" unchecked.
And then I uploaded the same file with a new name, where I checked the "Files can be shared with people external to my organization" option.

The only difference the 2 files had in the FILES.MEDIA table was this:

So, the column "SHARING_INTENT" is 0 on the file which I did not check the checkbox on, and it´s set to "1" on the file which I selected to share externally.

I then did a test in the Connections gui, where I tried sharing the "unshared" file with an external user. Result was that the external user did not show up in the gui.
I then performed the following SQL query to update the record, setting the "SHARING_INTENT" to the value 1:
update "FILES"."MEDIA" set SHARING_INTENT=1 where LABEL = 'kasper.docx';
And then I tried the same manoeuvre; tried to share the file with an external user, and this time, the external user came up in the type-ahead share box.

Is this enough? Are there other tables that also need to be updated?

I´m not sure, but I will test this on a customer who has been asking for this feature for a long time now.

So, what I´m thinking of doing now, is either running this SQL query as a scheduled script:
Where I always set every file to have the "SHARING_INTENT=1" where the query is:
update "FILES"."MEDIA" set SHARING_INTENT = 1;
or, create a TDI job that does this for me.
In TDI I can run the job on all the files that has been uploaded today, for instance, so that not all files are edited every time.
Or, perhaps do a Date-check in SQL:
update "FILES"."MEDIA" set SHARING_INTENT = 1 where CREATE_DATE > (current date); 

And also, I have created a PMR to ask for this feature. I asked IBM why the Cloud version has this option set automatically, and not the on-prem version. And I also asked about the fact that the Desktop Plugin don´t have this option at all.
I´m eager to learn why.

IBM Champion, Sharon Bellamy James, reminded me on Skype just now, that if users upload a file into a Community which is set to be shared externally, then this is not an issue. The files then gets automatically ticked with the "shared to eternal users" checkmark.
The issue is where you upload files directly into you own personal "FILES" and then share it to an external user or an "external community".

Thursday, November 10, 2016

Automate startup of IBM Docs 2.0 server - the correct way?!


I´ve done 5 different customer installations on IBM Docs 2.0 now, and they have all been having the same issues.
When I´ve done the installations, I´ve also created a scheduled restart of the server once a week. And after the server has restarted, the issues have been:
  • Viewer does not work as it should
  • Docs don´t work as it should
  • Thumbnail generation won´t work at all.
IBM initial setup, which is working fine, is that the scheduled tasks are set up to start on a "one time" trigger, and than with a 5 minute repeat interval.

Btw, do not alter this interval!! I tried... don´t do it.

And the scheduled tasks that are installed has a "when user is logged on" criteria.

When I have tried to change this to "wether or not user is logged on", and set the trigger to "on startup". in order to make the sched tasks start up automatically at windows reboot, without having to actually log on to the servers desktop... things was not working as expected.

The environments for the customers has been basically the same setup:
  • 1 IBM Connections server 
  • 1 Docs server where all three parts are installed (docs, viewer, conversion)
  • 1 DB2 server

My first installation was on a test server, where IBM Connections, DB2 and the IBM Docs servers were all installed on the same windows machine.
This server has no issues at all.

On the customer scenarios, where the Docs parts are installed on a separate server, a colleague of mine, @kilotin, created a startup script for the Docs parts. This script is executed through a Windows Scheduled task, and is triggered "on system startup".
The mounting of the IBM Connections shared folder is also something that´s called from within this script.

He also created a "stopdocs.bat" script, which shuts down the WAS server, node agents and disconnects the network drive mapping. It also does a reboot of the operating system.
(This script is shared later down in this blogentry, a modified version, in order to make this whole thing work.)

And, in order for the two Docs scheduled tasks ("sym_monitor" and "kill_timeout") to start automatically when the Windows server reboots, I´ve also modified those tasks to be triggered "on system startup" and to be executed with the option "start wether or not user is logged on".

All sounds good, right?
Well, not quite.

When I was in the process of developing this auto-start routine, this is what I did:
  1. Executed the "stopdocs.bat" script.
  2. Watched as the Websphere servers were shut down
  3. Then monitored that the OS was actually restarted.
  4. Logged into the desktop of the server, to monitor that the WAS processes were launched automatically.
  5. Then I tested each IBM Docs component manually (Create a document, edit, publish, print to pdf, uploaded documents, edited, watched as the thumbnail was generated.. etc, etc.)
Conclusion: Docs server is working fine and the restart routine works flawlessly also.


When a week went by, customers started complaining that the viewer didn´t work.
Especially documents of elderly Microsoft word format, ".DOC",  was not converting at all.
Newer document format, such as ".DOCX" was working....

SystemOut.log said stuff like:

[9/23/16 9:50:31:821 CEST] 0000047f QuerySnapshot E   CLFAF409W: Snapshot generation failed. Conversion error occurred. Error code:1203 Error message:Server returned unexpected status. Document id:ffbb67a7-5a98-49c8-8af7-28a74f86bc60 Document mimetype:application/msword Document version:1

And thumbnails previewing the content of the file is not shown either:

SystemOut.log said:
[11/8/16 19:51:46:095 CET] 00000132 UploadConvers I   CLFAF007I: Need to start a new conversion. This is for upload service. Doc id is 414ba791-1e60-4325-8e1d-d4f397b5014c. Mime type is application/vnd.openxmlformats-officedocument.wordprocessingml.document[11/8/16 19:51:46:188 CET] 00000132 ThumbnailServ I   Aquired thumbnail service lock successfully. DocId: 414ba791-1e60-4325-8e1d-d4f397b5014c LastModified: 1,478,631,105,495[11/8/16 19:51:46:188 CET] 00000132 ThumbnailServ I   CLFAF007I: Need to start a new conversion. This is for thumbnail service - one page upload conversion. Doc id is 414ba791-1e60-4325-8e1d-d4f397b5014c Mime type is application/vnd.openxmlformats-officedocument.wordprocessingml.document LastModified is 1478631105495[11/8/16 19:51:46:188 CET] 00000155 LCFilesEJBRep W   Can't find the file folder on EJB server! E:\shared\files\upload\files\35\65\eb4e2658-3cdb-4d5d-b79e-fe5637db363f[11/8/16 19:51:46:188 CET] 00000155 LCFilesEJBRep W   CLFAF703E: Get download document file. The doc id is 414ba791-1e60-4325-8e1d-d4f397b5014c[11/8/16 19:51:46:188 CET] 00000155 DocumentServi W   Image conversion for uploading failed. Document Id:414ba791-1e60-4325-8e1d-d4f397b5014c null

I then logged into the server, tested and then I see that ".DOC" documents started working again in the viewer. This was often after I did a manual restart of the Websphere Docs servers.
Thumbnails generation however, did not work.

Weird, right?

I then turned off the auto-restart scheduled task for each customer, while I was trying to figure this issue out. I made sure that the servers was running fine whilst doing so.

I tried everything I could think of. I created Windows services for the Docs WAS processes, ran those as an admin user, also tried as the SYSTEM user.... I tried mapping the network drive as different users, I was also in contact with IBM Champion Roberto Boccadoro, where we compared Docs configuration files with what he had in his system.... I then asked Roberto which user does he start the servers with. He was using "System". I then tried this on the scheduled task that starts up the servers as well.... Nothing gave results.

The thing that seemed to work, was actually ALWAYS logging manually into the Windows Desktop of the server.
That triggered stuff to work regarding .doc documents at least.

So, I then created a PMR, explaining what was going on. And after a few failed attempts, we were actually on to something!

IBM said that "you can´t use an Admin user in the Scheduled Tasks. You have to use the System user as the user that fires the Sched-tasks.". That means ALL THE TASKS, not only the Sched Task that starts the Websphere processes, but also the "sym_monitor" and "kill_timout" process.

And also, IBM also reccomends calling the Mount script in a separate script....... (which is wrong... I´ll discuss this later on).

Ok, first of all. When you install IBM Docs, the two scheduled tasks "sym_monitor" and "kill_timeout" is created with the user you are logged in as the installation was running. So, those two Sched tasks I had to manually set to run as "System".

I then restarted the Docs Windows server, and tested without logging into the desktop of the server this time.

Wow, now ".DOC" documents was actually working. Converted just fine and the viewer worked.

But one thing was still not working, and that was the thumbnail creation of the document.

I then stumbled upon an article (WHICH I DID NOT BOOKMARK and can´t find it again), describing the fact that when you map a drive in a script through a scheduled task, throug a "call" to a different mount script (as IBM suggested), and when this process is started up as the "SYSTEM" user, there is no sharing between the two processes.
Which means, the system user that starts up the WAS servers does not have access to the mapped drive, which was called through a separate mount script!!

So the solution, is to do the drive mounting inside the same script that starts up the Docs WAS processes. This is opposed to what IBM suggested.

So... This has been a long read for you guys. Here´s the fun part: How to correctly automate the startup of the Docs server and all it´s related processes:

First, you need to create this script:

(The order of starting up the WAS servers are important; 1. Conversion, 2. Docs, 3. Viewer.)

1. Create a directory e.g. D:\Scripts
2. Create a new subdirectory e.g. D:\Scripts\logs
3.Create file in D:\Scripts with the name "mapAndStartDocs.bat" with this content:
(Modify the script to suite your environment)
(I only map 1 drive, the Connections Shared Folder share, The docs_share and the viewer_share is locally on the Docs Windows server, which means I don´t need to map those to u: and v:)

@echo off

echo Mapping IBM Connections Shared Data disk %date% %time% > D:\Scripts\logs\mount.log
net use E: \\appsrv1.skya.local\e$\IBM\Data /user:USERNAME PASSWORD /persistent:yes >> D:\Scripts\logs\mount.log

echo Start conv nodeagent %date% %time% > D:\Scripts\logs\startNodes.log
call D:\IBM\WebSphere\AppServer\profiles\conv1Node01\bin\startNode.bat >> D:\Scripts\logs\startNodes.log

echo Start docs nodeagent %date% %time% >> D:\Scripts\logs\startNodes.log
call D:\IBM\WebSphere\AppServer\profiles\docs1Node01\bin\startNode.bat >> D:\Scripts\logs\startNodes.log

echo Start viewer nodeagent %date% %time% >> D:\Scripts\logs\startNodes.log
call D:\IBM\WebSphere\AppServer\profiles\viewer1Node01\bin\startNode.bat >> D:\Scripts\logs\startNodes.log


echo Start conv server %date% %time% > D:\Scripts\logs\startServers.log
call D:\IBM\WebSphere\AppServer\profiles\conv1Node01\bin\startServer.bat IBMConversionMember1 >> D:\Scripts\logs\startServers.log

echo Start docs server %date% %time% >> D:\Scripts\logs\startServers.log
call D:\IBM\WebSphere\AppServer\profiles\docs1Node01\bin\startServer.bat IBMDocsMember1 >> D:\Scripts\logs\startServers.log

echo Start viewer server %date% %time% >> D:\Scripts\logs\startServers.log
call D:\IBM\WebSphere\AppServer\profiles\viewer1Node01\bin\startServer.bat IBMViewerMember1 >> D:\Scripts\logs\startServers.log


Then create the script stopDocs.bat and put it in the same D:\Scripts folder.
stopDocs.bat content:

@echo off

echo Stop viewer server %date% %time% > D:\Scripts\logs\stopServers.log
call D:\IBM\WebSphere\AppServer\profiles\viewer1Node01\bin\stopServer.bat IBMViewerMember1 -username WASADMIN -password PASSWORD >> D:\Scripts\logs\stopServers.log

echo Stop docs server %date% %time% >> D:\Scripts\logs\stopServers.log
call D:\IBM\WebSphere\AppServer\profiles\docs1Node01\bin\stopServer.bat IBMDocsMember1 -username WASADMIN -password PASSWORD >> D:\Scripts\logs\stopServers.log

echo Stop conv server %date% %time% >> D:\Scripts\logs\stopServers.log
call D:\IBM\WebSphere\AppServer\profiles\conv1Node01\bin\stopServer.bat IBMConversionMember1 -username WASADMIN -password PASSWORD >> D:\Scripts\logs\stopServers.log

echo Stop viewer nodeagent %date% %time% > D:\Scripts\logs\stopNodes.log
call D:\IBM\WebSphere\AppServer\profiles\viewer1Node01\bin\stopNode.bat -username WASADMIN -password PASSWORD >> D:\Scripts\logs\stopNodes.log

echo Stop docs nodeagent %date% %time% >> D:\Scripts\logs\stopNodes.log
call D:\IBM\WebSphere\AppServer\profiles\docs1Node01\bin\stopNode.bat -username WASADMIN -password PASSWORD >> D:\Scripts\logs\stopNodes.log

echo Stop conv nodeagent %date% %time% >> D:\Scripts\logs\stopNodes.log
call D:\IBM\WebSphere\AppServer\profiles\conv1Node01\bin\stopNode.bat -username WASADMIN -password PASSWORD >> D:\Scripts\logs\stopNodes.log

echo Unmount share %date% %time% > D:\Scripts\logs\unmount.log
net use * /delete /yes >> D:\Scripts\logs\unmount.log

echo Reboot %date% %time% > D:\Scripts\logs\reboot.log
shutdown -r -t 3

Then, change the two scheduled tasks like this:
Set the user to "SYSTEM", run with highest priv and "hidden"

Change trigger to "At startup" and 5 min repeat and duration to Indefinitely:

Do the same for "sym_monitor":

Then, create a new Scheduled task with the name "StartDocs".
Set the configuration as the following screenshots show:

Then, if you want to schedule a restart of the Docs server, create a Scheduled task with the name "StopDocs"

You can then test the restart out by right-clicking the "StopDocs" scheduled task and select "Run."

If all goes well, the WAS servers and NodeAgents will shut down, and a windows reboot will occur.
And when the server is back up again, you will be able to create Docs documents + upload files with .doc format and preview them + you will notice that the Thumbnail generation of the files are working as well.

Phew... This has been a huge headache... I´m thrilled that I´ve come up with the solution on how to automate the startup of the Docs servers with a scheduled reboot of the servers.


P.S. Any comments or questions or suggestions on improvements, please post them in the comments section.