ICS/Lotus (mostly), Linux, Travel, Skiing, Mixology, and Random Musing of Interest

The 2015 Linux Jobs Report Is Out -- Strong Demand Continues

Bill Malchisky  March 5 2015 04:42:27 PM
"Unstoppable Linux Job Market Shows No Signs of Slowing Down"

The Linux Foundation in cooperation with Dice released their March 2015 Linux Jobs report. In a phrase, growth for Linux professionals remains strong once again and is growing stronger. If you are looking for a new opportunity, or to expand your existing skill set, Linux is a great way to do that. As I reported in 2013, the trend line for Linux talent progressed upward from 2012 through 2013 and that remains true today. Cloud services, mobile devices, application servers, perimeter devices, security tools, smart appliances all use Linux. In fact, most pro-Windows techs utilize Linux in their daily lives and just may not realize it. The market permutations are expanding due to its quality, ease of use, and ability to work with just what you need for your task at hand (e.g. micro-installations). Plus, the price is more than reasonable too.

This year's report is available here. It is free, but you do have register for the download.
The Linux Foundation's press release is available here.
Dice's press release page is here.

Three key bullet points from the report
1. Nearly all hiring managers are looking to recruit Linux professionals in the next six months
2. The rise of open cloud platforms is creating even more demand for Linux professionals with the right expertise
3. Linux-certified professionals will be especially well positioned in the job market this year

Remember, "Employers are hungrier than ever for Linux talent." (Page 3). Don't get left behind: learn Linux and stay marketable.

Training Available
If your or your company is interested in training your ICS administrators on how Linux can work with your company, or to learn how your company save money by using Linux on some of your servers, please let me know. I have an extensive track record with Linux training and would thoroughly enjoy helping people learn Linux, and provide a perspective on decreasing costs. Thank you for your interest.

"Age is Just a Number..." -- George Jedenoff; 97.5 Year Old Skier In Another Inspriational Interview

Bill Malchisky  February 19 2015 11:00:00 PM
Last year, I commented on George Jedenoff and how much he loves to ski. Well, the passionate 97 -1/2 year old skier returned for another year of carving at Alta, Utah and did another interview with the Ski Utah crew. The adventurous Mr. Jedenoff skis trees, has a great relaxed form, and loves Wasatch powder. This year, he answered several questions from Ski Utah readers and provided fun candid answers, on longevity, diet, how he preps for each ski season, and how he finds ski buddies. An inspirational four minute video that may just make you smile.

"Age is just a number. If you let the number bog you down, you are going to bog down. You forget about the number, and live every day with what you got, you are going to last longer." --George Jedenoff

IBM ConnectED (LS15) Saturday Community Events Time Changes

Bill Malchisky  January 24 2015 11:38:36 PM
Due to the overnight and morning rain, I started working on logistical changes to get the day's Community events included with minimal overlap to other already scheduled Saturday events.

This should serve as the Master Schedule for now... as due to traveling to this event, the wonderful teams for the Totally Unsupported IBM Notes Session Database  and  the AngularJS and Domino demo site. by Mark Roden and Mark Leusink, will not be able to get their sites updated in-time, which is completely reasonable and inappropriate to ask with such short notice.

Saturday, 24 January 2015
Soccer Saturday - 11:30 AM to 1:30 PM
BALD - 2:00 PM (ish) to 5:30 PM
ESPN 7:30 PM til 11:30 (Unchanged)

Note: All locations remain the same.

There is a lot going on today (24 January) outside of these events. As such, I am happy that we can work with Mother Nature and still enjoy our day with friends before the event commences on Sunday.

My IBM ConnectED Speaking Schedule -- With My First ChalkTalk

Bill Malchisky  January 21 2015 08:45:00 PM
Image:My IBM ConnectED Speaking Schedule -- With My First ChalkTalk

In just a few short days, the next iteration of our annual January conference commences. With all of the other ICS related activities I am doing this year outside of IC15 (LS15), I am happy to take a bit of a back seat with the speaking and run my first ChalkTalk session (formerly Birds Of a Feather (BOF)), plus Linuxfest VI on Wednesday.

For my ChalkTalk session, I am co-presenting with the wonderful fellow IBM Champion, Ms. Femke Goedhart. We are covering the Personal Privacy Paradigm. A timely topic with all of the data breaches being reported in the news. We both bring years of personal and consulting experience to this topic, and recently collaborated on an data privacy project. We are both thrilled to be able to present this topic with many useful examples. We think you will find the interactive discussion both lively and informative. See you there!

Date: Tuesday, 27 January
Time: 6:15 - 7:00 PM
Place: Swan -- Swan 7-10
Audience: Cross Industry
Speakers: Bill Malchisky, Femke Goedhart


In a world of ubiquitous information access, how does one maintain their privacy and avoid future forward situations that may be hard to extrapolate? People have a right to privacy, but through the combination of lawyers and technology what used to be an easy paperwork process can introduce significant gray areas with overreaching background check releases and agreements that can be unwise to sign as is. Additionally, customers need to protect their information, regulated data, and trade secrets, all while ensuring outside vendors can do their job even when working through multiple layers of contracting organizations. This BOF discusses trends in privacy protection and degradation while providing tips you can use to help you with that next gig while also protecting yourself, your family, or your company. Please bring your privacy challenges as Femke and Bill provide insight while fostering what should be a lively group discussion

IBM ConnectED -- Get your Linux Fix at Linuxfest VI

Bill Malchisky  January 21 2015 12:57:15 AM
 Image:IBM ConnectED -- Get your Linux Fix at Linuxfest VI

After several months of planning, I can officially announce that Linuxfest will return to Orlando, for IC15 for its sixth consecutive year. With the compressed schedule this year and the organizing team working diligently to make the event memorable for all the speakers and attendees, we had to schedule this unofficial session differently. Thus, we are in the Swan and one-half hour after the CGS (Closing General Session).

We are listed in the Totally Unsupported IBM Notes Session Database as session ID COM400, in the Community Track. If you do not have this application, it's a worthwhile application. Ensure you synchronize it daily to get all the latest changes from the overnight minions hard at work adding content and ensuring its accuracy.

Date: Wednesday, 28 January
Time: 4:30 - 5:30 PM
Place: Swan -- Poolside Bar
Audience: Admins, Developers, Architects
Speakers: Bill Malchisky, Wes Morgan, and Daniel Nashed

This is the only session dedicated exclusively to Linux. After the CGS and some photos with friends, head directly to the Swan pool area. There is no post-event activity this year, so take a few minutes to get tasty beverage, and let's talk Linux. Ask questions and get informative answers from three passionate leading IBM on Linux SMEs.


Do you have Linux questions? We have answers. This most informative IBM-focused Linux session returns in our sixth year! Join Bill, Wes, and Dan for community knowledge sharing on all things ICS/Linux related. Due to the shortened schedule this year, we are in a new spot and time at IBM ConnectED. Please join us :30 after the CGS by the Swan Poolside Bar. See you there!

Get Access -- Voice of the Partner is Live

Bill Malchisky  December 17 2014 02:00:00 AM
It is with great pleasure that I announce Voice of the Partner is in production! This tool provides IBM Collaboration Solutions business partners a new feedback continuity site to interface directly with IBM. As alluded to within my beta launch blog post, the past three months proved quite busy, culminating in this promised mid-December launch.

Please understand this is a volunteer effort for me and the scope of BPs in good standing is massive. Thus, I am breaking the overall group down into three waves. The first group includes over 100 unique companies that are active within the community through sponsoring IC15, sponsored user group events recently, and BPs that have IBM Champions. That set will keep me busy for at least a few of days as I send announcement e-mails, then process those responses with my IBM contact. Once on-board, the ensuing two waves will be constructed from received requests and member suggested companies, the tertiary wave being on-going. To help ensure outreach to the greater ICS business partner community, IBM's relevant social media accounts and the next couple of BP newsletters will be mentioning the tool.

How to Join

If your company would like to be involved and gain access to this excellent environment, please perform the following three steps:
1. Select up to two people to represent your company (e.g. Administration and development issue contacts; but can be whatever you want)
2. Send me an e-mail via this link with the name(s) and e-mail address(es) utilized for IBM Connections Cloud (formerly SmartCloud)
3. Optional: Indicate if you want Sametime awareness on each account, thus allowing partners to IM members; omitting this value equates opting-out

A. Altering the subject will delay your processing significantly
B. Succeeding access is a queue: the sooner you request it, the sooner you will receive it
C. Your access confirmation will be sent via IBM Connections Cloud site
D. It is expected that ICS BPs can and should submit relevant content on-behalf of their customers--therefore strengthening that relationship while maintaining discretion
E. If you need to swap a name in the future, use the above e-mail link with the message body containing the name to demote and the name to promote

What You Can Expect

* One big change since the beta announcement, is that we are now utilizing IBM Connections Cloud
* The Connections Community's Overview page explains the overall workflow--please adhere to it
* The Dashboard is version 1.0, data will be updated quarterly, and evolved over the next two quarters
* IBM is building an internal framework to properly assign issue owners and ensure a proper response within a reasonable time-frame; please be patient as we work with them

If you have any questions, please feel free to leave a comment below, contact me via Twitter or Skype, or privately here.

This project is a tremendous benefit to the ICS BP community and I want to thank IBM for their outstanding support and commitment at the highest levels to ensure success.

IBM Mobile Connect POODLE Fix

Bill Malchisky  November 3 2014 02:45:00 AM
IBM just announced a couple of Technotes dealing with IBM Mobile Connect and POODLE. As this product is quite secure by design and a product that I enjoy as customers have a great track record with it in the field, the option to use TLS 1.0 - 1.2 is supported. The new SSL v3 security changes are implemented under APAR IV66131 -- available for IMC 6.1.5 and Thus, if you have not upgraded IMC in the past ten days, then read below and plan your upgrade as appropriate.

Here Are The Technotes
How is IBM Mobile Connect impacted by the POODLE attack?
Configure IBM Mobile Connect to disable SSL V3 ciphers - this includes detailed particulars on the new SSL v3 command set to toggle the feature
APAR Fix List - APAR IV66131 addresses the implementation of these fixes, which disables SSL v3 by default, and commands to toggle this feature for internal or external connections; build date 22 Oct 2014
IBM Mobile Connect Maintenance Releases
Additional documentation for IMC 6.1.5 or upgrading IMC 6.1.4 to 6.1.5 is located here

1. To view your current IMC version, type: #lswg -V | more  --or-- use the Gatekeeper -> About Tab -> Connection Manager properties
2. These APAR fixes are for Connection Manager only and omit the Gatekeeper and Mobility Client in the current code stream, by design

IBM Protector for Mail Security POODLE Fix

Bill Malchisky  October 22 2014 12:05:00 PM
A day after providing two Technotes on SHA-2, TLS, and POODLE for Domino, IBM released two documents to cover their Protector product. The bulletin covers three vulnerabilities and provides details on each. For the workaround document, mind the side effect mentioned at the bottom, as with some sites, this may introduce a risk assessment against delivery versus the exploit's vulnerability.

1. Bulletin - Security Bulletin: Vulnerabilities in OpenSSL may cause weak cyphers to be used over SSLv3 (POODLE Attack) in IBM Lotus Protector for Mail Security (CVE-2014-3513, CVE-2014-3567, CVE-2014-3568)
2. Suggested workaround - How to protect Lotus Protector for Mail Security against POODLE (Padding Oracle On Downgraded Legacy Encryption) attack

: It should be noted that the bulletin omits CVE 2014-3566, the primary exploit for POODLE; as per IBM Tech Support they are specifically releasing an HTTPS patch for this exploit--via a PMR opened by a customer.

Hat tip to Samuel Sawatzky for the heads-up.

Silent No More: IBM Makes Security Announcements on SHA-2, TLS, POODLE

Bill Malchisky  October 21 2014 08:45:00 AM
Today (21 Oct 14), IBM created a set of Technotes covering what appears to be a first step in helping soothe the customer and partner concern on the lack of offered direction and plan for resolving the SHA-1, TLS, and POODLE exploits that exist from years of community support and a yet to be implemented capability for increased security. I offer first step as no date for the patch is provided, just that they are stating their intentions and scope with a solution by year-end, which is my conjecture derived from their "several weeks" window statement. Recall that Google forced the hand by what appears to be an arbitrary cut-off for accepting SHA-1 SSL certificates in their browsers (and exempts customers who buy their SSL certificates from Google, I will add).

With IBM responding now, customers, partners --- including ISVs --- can now plan accordingly. Happy to have these documents. Thank you, IBM.

Here is what IBM offered

1. How is IBM Domino Affected by POODLE?
2. Planned SHA-2 Delivers for IBM Domino 9.x
3. As people will undoubtedly ask, Is it Possible to Run IBM HTTP Server (IHS) on the Same Computer as a Domino Server?


1. These SHA-2 fixes are for ND9 only, and do not work with 8.5.3 due to changes in the security model inherent to each build
2. The POODLE fix goes back to D8.5.1FP5
3. They are covering all the appropriate Internet protocols that your customers use

"With this Interim Fix, Domino administrators will be able to configure Domino 9.x to use a SHA-2 certificate over HTTP, SMTP, LDAP, POP, and IMAP. With a SHA-2 certificate in place, users will be able to use a browser to connect to iNotes, XPages, traditional Domino Web apps, and Sametime (based on Domino HTTP)."

New SSL3 Exploit: The POODLE Is Here and Lifting Its Leg

Bill Malchisky  October 15 2014 03:12:00 AM
Fix Update -
{Update 9, 17 Oct 2014} As companies are working day and night to get a fix for this product, I will keep checking and update you here. Red Hat has a few patches for their various OS releases but they are reported as incomplete and thus not formally released yet. The rest of the post has been updated eight times since initial publication and is pretty static now as we are all in a wait-state until a patch is formally approved and released.
{Update 10, 17 Oct 2014, 10:01 PM} Starting to see smaller client side packages with POODLE fixes being released.
{Update 11, 18 Oct 2014 2:43 AM} Server-side partial fix announced
See the new Impact section below for the two attack permutations, their impact, and risk, plus links to the RHEL version specific package errata in "Fixing the Exploit" at the bottom

Now we are just waiting for primary exploit fix and client-side browsers to ensure that you have a full and complete solution. This gets you one-third of the way there.

Here we go again... another blockbuster security exploit with another clever code name is announced. POODLE (Padding Oracle On Downgraded Legacy Encryption) CVE-2014-3566 specifically allows a man-in-the-middle style attack utilizing an SSL3 connection. Once again, Red Hat does a stellar job offering full details on background, technical specifics, and testing. Google's Online Security Blog post is exceedingly terse when contrasting. Here is what you need to know.

What is It?

CVE-2014-3566 allows one to decrypt ciphertext using a padding oracle side-channel attack. Look at it this way... this exploit sees every SSL server as a fire hydrant and the man-in-the-middle style POODLE has been drinking water for a long time. The scope is bigger than Shellshock, in my opinion, as it hits any server running SSL 1.0, 2.0, 3.0, and TLS 1.0, plus client-side applications like browsers, but the damage impact appears to be less that one can cause with Shellshock as there is not a remote code execution capability and your system would have some capacity to be productive. To that point, it is (or should be) a high priority for any vendor who hosts SSL and is listed as a high impact
Red Hat published a Security impact key.


It is categorized as a High priority and High Severity, which is the third highest of the former and second highest on the latter. So, although this is a nasty exploit, the damage to your systems could be worse. In summary, take care of it, but do not panic.

Red Hat is making this a top priority and has a KB article (#1232123) on the subject. Excerpts are below. Also, Google's security blog has a couple of paragraphs on this exploit and an eye towards a patch for their products.


{Update 10 - 17 OCT 2014; new section}
Two Attack Permutations
1. Man-in-the-Middle: To quote Red Hat on this exploit's impact, they say, "Exploiting this vulnerability is not easily accomplished. Man-in-the-middle attacks require large amounts of time and resources. While likelihood is low, Red Hat recommends implementing only TLS to avoid flaws in SSL." -- Red Hat via Knowledge Base Article 1232123;
2. Fallback Attack -- correlated secondary exploit, explained below.

Red Hat (among others) indicated a TLS Fallback option to let client side applications (e.g. Browsers) to inform a server that they can handle the newer SSL/TLS versions (safer ones). Here is the issue, not all browsers support this capability. So rather than being dropped, a dubious connection attempts to revert to a lower protocol version when the server supports a newer secure protocol version, the unsupported browser flavors allow the insecure protocol connection--creating a POODLE exploit.

Chromium is the only Linux graphical browser that supports the fallback attack security. Once enabled on a server, the client must support it as well. Firefox and Curl (tui browser) are vulnerable at the time of this writing.

Additional concerns

As a workaround, it has been suggested to disable SSL and utilize TLS until a proper SSL patch is released. This of course works for HTTPs client tools that support TLS. If the tool does not understand TLS and you disable SSL, then you have zero security which is much worse than having a POODLE vulnerable SSL version in production.
Therefore, it is quite important to understand your use case needs before implementing any workaround.
Technical details on the browser attack are located here.

A fix for the openssl packages that addresses the Fallback attack is available; the primary SSL3 exploit remains open. See "Fixing the Exploit" section below.

Testing for the Vulnerability

Three convenient tests exist to verify the status of concerned servers.
1. Run this command on your server or remotely if easier to see if your server is vulnerable. This does not text specific applications on said server that may be configured to use SSLv3. Change the "$(hostname)" value to a FQDN.

malchw@san-domino:~$if echo Q | openssl s_client -connect $(hostname):443 -ssl3 2> /dev/null | grep -v "Cipher.*0000"; then echo "SSLv3 enabled"; else echo "SSLv3 disabled"; fi


Negative: SSLv3 disabled
Positive: SSLv3 enabled

2. Red Hat Access Lab created a SSLv3 (POODLE) Detector  GUI testing tool including a browser check, a BASH shell script to check servers offline, and a realtime check for public facing-servers via a FQDN and port of your choosing--launched from their network. You will need an RHN account to access it. I like the third option with Red Hat's tool because it can make easy work allowing customers while at work to check outside their office with ease.

3. If you want a more verbose output, try the original test here
malchw@san-domino:~$ openssl s_client -connect localhost:443 -ssl3

malchw@san-domino:~$ openssl s_client -connect [hostname.foo.com]:443 -ssl3

Note: Change the port number and hostname to suite your specific test case.

Positive result

If you see something similar to this, you are vulnerable
Results - New, TLSv1/SSLv3, Cipher is DES-CBC3-SHA

Server public key is 1024 bit

Secure Renegotiation IS NOT supported

Compression: NONE

Expansion: NONE


Protocol  : SSLv3

Negative Result

Else, you are fine, if this excerpt is close to your output:
140128201074504:error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure:s3_pkt.c:1257:SSL alert number 40

140128201074504:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:596:


New, (NONE), Cipher is (NONE)

Secure Renegotiation IS NOT supported

Compression: NONE

Expansion: NONESSL-Session:

I suspect most servers with SSL will have SSL3 enabled, making the situation more wide-spread than some people may realize. This is operating system agnostic, so Microsoft, IBM, Red Hat, Oracle, VCE, et alia will all have upcoming product lists with affected products and tools, needing attention.

Fixing The Exploit
-- {Update 18 Oct 2014 - Patches released}
Fix Announced
-- 1 of {n}
Red Hat offers a fix for POODLE's secondary Fallback attack exploit is available under advisory: RHSA-2014:1652-1 for RHEL 6, 7 plus address two memory leak fixes that can cause a crash; and RHSA-2014-1653-1 for RHEL 5.
After patching, be certain to restart the httpd,ssld, and all other SSL-enabled services running on your box. And patch your box prior to applying this patch, to ensure success. Details from OpenSSL are available in their POODLE security advisory.

"The openssl packages errata linked [above] do not address the SSL 3.0 protocol issue, to which the CVE-2014-3566 was assigned.  They add support for the TLS_FALLBACK_SCSV, which enables TLS servers to detect forced protocol downgrades against clients that do re-connect protocol version fallback.  Both server and client need to implement this feature, and clients have to actively use it for the protection to work." --Tomas Hoger, Red Hat Product Security

If you need assistance in applying package updates from RHN, see article 11258.

Further details on the main link provided as made available.

{Update - four times this morning, plus early evening, 15 Oct, EDT}
At this time a proper fix is unavailable. As of midnight EDT on Wednesday, Red Hat has not released a fix yet, but is working on it. Other companies will need to do the same, as well as browser ISVs to ensure compatibility. In the mean time, the suggested work-arounds are as follows:
1. Disable SSLv3 on your servers
2. Even if impractical to do disable SSLv3, consider using TLSv1.1 and TLSv1.2 with the  TLS_FALLBACK_SCSV parameter on your TLS servers enabled (Internet white paper draft available). This process may cause a few issues with some IBM products (to be fair, most vendors products will have issues even temporarily)
3. If you run nginx, here is a solution: https://news.ycombinator.com/item?id=8456178
4. If you are running Domino, you need a TLS 1.1+ server in-front of it, as IBM has not provided a solution; fellow IBM community member, Darren Duke has a work-around with Apache on Ubuntu utilizing TLS 1.2 (and v1.1) for advanced users
--The problem with Domino as it is now is that if you remove SSLv2 and SSLv3, then you need a new TLS solution in-front of that server for protocols such as HTTP, SMTP, et alia.
5. As Domino is directly impacted, please add your voice here as well

{Update - 12:08 PM EDT}
6. As Craig Wiseman reminded me, there are a few great (but hardly trivial) nginx workarounds published by the community
Jesse Gallagher: Domino and SSL: Come with Me If You Want to Live
Richard Moy: Installing Nginx Reverse Proxy on CentOS for Domino Our Experience
Ray Davies: Domino Interface: Installing Nginx Reverse Proxy on CentOS for Domino Our Experience
{End of Update}

Red Hat's in-scope products (at this time) are here:
Affected Component(s)
Red Hat Enterprise Linux Tomcat, Firefox/Chromium, httpd, OpenSSL
JBoss Enterprise Middleware Tomcat/JBoss Web, httpd, OpenSSL
Red Hat Network Satellite Tomcat
Red Hat Certificate System Tomcat
Inktank Ceph Enterprise httpd
Red Hat Enterprise OpenShift OpenShift Configuration , RHC client tools
Red Hat Enterprise Linux OpenStack Platform httpd
Red Hat CloudForms httpd
Red Hat Directory Server Directory Server Configuration
Red Hat Enterprise Virtualization RHEV-M


1. Each affected component's hotspot offers a product specific technote on how to address the fix for the specific product and more focused testing too;
2. The table is current at the time of this writing and may expand as a fix is released by Red Hat and other products identified
3. To their credit Red Hat is working constantly to update their product list and resolution guides, adding the four appended rows to this table, early evening, 15 Oct 2014 EDT; and now a fifth for Enterprise Virtualization, late afternoon, 17 October 2014 EDT

Google provided a white paper entitled, This POODLE Bites: Exploiting the SSL 3.0 Fallback, which provides greater detail on the TLS suggested settings and the exploit itself. Google is suggesting the use of TLS_FALLBACK_SCSV too.

More later when a fix is released. Good luck.

Powered by IBM Lotus Domino 8 | Lotus User Group | Get Firefox! | This blog is listed on Planet Lotus   IBM Certified

© 2010 William Malchisky.