Red Hat

Serial console support in Grub2

This is another one of those “just so I don’t forget” posts. I’m working towards migrating from IPA in a KVM-based virtual machine running CentOS 6 to one running CentOS 7 and the way of making serial consoles work in 7 is different than it was in 6 because CentOS 7 (and thus RHEL7) use Grub2.

Instead of editing /boot/grub.cfg, you need to edit /etc/defaults/grub and add:

GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200"

The above will enable Grub output to the serial console. Next, change the GRUB_CMDLINE_LINUX command to the following so you can use “virsh console” to connect to the guest:

GRUB_CMDLINE_LINUX="rhgb quiet console=tty0 console=ttyS0,115200"

Once you have saved the file, you need to recreate the grub.cfg file with grub2-mkconfig:

# grub2-mkconfig -o /boot/grub2/grub.cfg

Now, reboot the system and you should be able to use “virsh console [VMNAME]” to have a serial console to the guest.

Getting started with firewalld

I’m mostly writing this for my own reference as I spent a bunch of time figuring this out while I was on holidays with some serious oVirt misadventures and didn’t document any of what I did, so since I had to reinstall CentOS 7, I’m stuck doing this all over again.

Effectively I’m migrating from CentOS 6 to CentOS 7 and trying to take advantage of the new way of doing things. I could easily switch things to use /etc/sysconfig/iptables (that I can handle pretty much in my sleep) but it feels a bit like cheating, so I want to figure out firewalld which is to me a totally alien way of handling my firewall rules.

Suffice it to say that there may be better ways of doing this (and, to you the reader, probably far better tutorials), but this is how I went about it. This is on a CentOS 7.1 system.

First thing is to make sure firewalld is running and will run. It should be the default but never hurts to check:

[root@thor dhcp]# systemctl status firewalld
firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled)
   Active: active (running) since Sat 2015-08-15 18:21:30 MDT; 18min ago
 Main PID: 745 (firewalld)
...

Ok, it’s active and enabled (in particular pay attention to the enabled bit that is bolded above) so on a reboot it will come up. firewalld has this concept of zones, so you can set certain ethernet interfaces to be assigned to a particular zone. The default zone is public, but as this is an internal DNS/DHCP/web server for my LAN, I want to set it to the internal zone:

[root@thor dhcp]# firewall-cmd --get-zones
block dmz drop external home internal public trusted work
[root@thor dhcp]# firewall-cmd --set-default-zone=internal
success

I also want to make sure that the one ethernet interface on this machine, enp2s0 (so long eth0!) is assigned the same zone. I believe this would happen by default, but it doesn’t hurt to be explicit:

[root@thor dhcp]# firewall-cmd --zone=internal --change-interface=enp2s0 --permanent
success

Now we can see how this zone is configured:

[root@thor dhcp]# firewall-cmd --zone=internal --list-all
internal (default, active)
  interfaces: enp2s0
  sources:
  services: dhcpv6-client ipp-client mdns samba-client ssh
  ports:
  masquerade: no
  forward-ports:
  icmp-blocks:
  rich rules:

Not a lot there right now and it’s definitely setup to be a workstation, not a server. You can see the available list of services that come pre-defined (they are XML files in /usr/lib/firewalld/services) by using:

[root@thor dhcp]# firewall-cmd --get-services

If you want to examine any of the listed services, just look at the XML file. For instance, /usr/lib/firewalld/services/dns.xml looks like:

<?xml version="1.0" encoding="utf-8"?>
<service>
  <short>DNS</short>
  <description>The Domain Name System (DNS) is used to provide and request host and domain names. Enable this option, if you plan to provide a domain name service (e.g. with bind).</description>
  <port protocol="tcp" port="53"/>
  <port protocol="udp" port="53"/>
</service>

Pretty straightforward. When adding new services or rules, firewalld makes them temporary, in that they will persist until a reboot or service restart. You need to use the “–permanent” option to, well, make them permanent. Since this server is going to do DNS/DHCP/HTTP/HTTPS we need to do:

[root@thor dhcp]# firewall-cmd --permanent --zone=internal --add-service=http
success
[root@thor dhcp]# firewall-cmd --permanent --zone=internal --add-service=https
success
[root@thor dhcp]# firewall-cmd --permanent --zone=internal --add-service=dns
success
[root@thor dhcp]# firewall-cmd --permanent --zone=internal --add-service=dhcp
success
[root@thor dhcp]# firewall-cmd --reload
success
[root@thor dhcp]# firewall-cmd --zone=internal --list-all
internal (default, active)
  interfaces: enp2s0
  sources:
  services: dhcp dhcpv6-client dns http https ipp-client mdns samba-client ssh
  ports:
  masquerade: no
  forward-ports:
  icmp-blocks:
  rich rules:

In the above we added four services: HTTP, HTTPS, DNS, and DHCP. We then reloaded the rules (so as to apply them) and then listed the internal zone where we can now see our new services are enabled.

If you want to enable a service for which no default service definition exists, you can create one in /etc/firewalld/services/ as an XML file (copy a similar one from /usr/lib/firewalld/services/ and adjust for your service). If you’re using SELinux, be sure to run restorecon -Rv /etc/firewalld and make sure the XML file has 0640 permissions and is owned root:root.

Instead of making a new service, however, you can simply add ports to the configuration which might be easier. For instance, if I wanted to expose apcupsd but didn’t want to make a service I might do:

[root@thor dhcp]# firewall-cmd --zone=internal --add-port=3551/udp --permanent
success
[root@thor dhcp]# firewall-cmd --zone=internal --add-port=3551/tcp --permanent
success
[root@thor dhcp]# firewall-cmd --reload
success
[root@thor dhcp]# firewall-cmd --zone=internal --list-all
internal (default, active)
  interfaces: enp2s0
  sources:
  services: dhcp dhcpv6-client dns http https ipp-client mdns samba-client ssh
  ports: 3551/udp 3551/tcp
  masquerade: no
  forward-ports:
  icmp-blocks:
  rich rules:

For further reading, I recommend:

In brief, managing the firewall is a little bit annoying to do this way compared to the old way of “vim /etc/sysconfig/iptables; service iptables restart” but I can understand why it’s done this way now. It’s quite modular and adaptable and allows you to make temporary changes to the firewall easily without having to know iptables commands. There are a lot of commands, and since firewalls tend not to be things you tinker with often, they may not be overly memorable (thus deciding to write this after my second time of doing it in as many weeks as I didn’t remember a darn thing).

Heartbleed

I’ve refrained from posting or saying anything about Heartbleed all week because I didn’t want to add to any sensationalism and hype, and I’ve also been too busy actually dealing with it (as opposed to simply talking about it or running around with hands waving in the air like a mad man). Now that the dust has settled a bit, I just want to link to some sites that I think are good to keep handy as we see this play out. I don’t need to talk about the flaw itself as all you need to do is google “heartbleed” and you’ll get all the info you want; certainly more than I can provide here (although you will have to distill the sensational from the facts).

So, the sites:

  • Heartbleed Bug Health Report; they’re keeping it up to date, but it’s essentially a “top 1000 still-vulnerable sites” list
  • Mashable’s Heartbleed Hit List which has a list of some of the bigger sites/services that were (or were not) affected and whether they still are; when I looked this morning it was last updated as of last night so presumably they’re keeping it fairly up to date
  • DigitalTrends Mobile app list which has a list of vulnerable/not-vulnerable mobile apps
  • The Heartbleed site which is being kept up to date with regards to linking to various advisories

Some of these sites (and the apps that use such sites) have been fixed this week. There is speculation that this has been known for a while which means the “window of opportunity” may be much bigger than was initially thought. Some of the numbers being tossed around are pretty gross exaggerations though (one I saw was “66% of the internet vulnerable!”) so you have to take things with a grain of salt. The best advice is to look at the sites you use and if they have fixed the flaw (and were previously vulnerable) and recommend you doing something (like changing your password), strongly consider doing as they suggest — PROVIDED THEY HAVE ALREADY FIXED THE FLAW! Sorry for the caps but I talked to some people yesterday who had rushed to change their password and when I asked them if the site in question was fixed already, they gave me a blank stare.

It does you NO good to change your password to a site that is STILL vulnerable. You will only have to change it again.

Anyways, look at the sites noted above, breathe, and keep in mind that changing passwords occasionally is a good thing. Maybe now is the time to start using something like LastPass, 1Password, KeePass, or something similar and having it generate pure random nonsense for a password, knowing that you can use this tool/service to remember it for you (although, arguably, this whole situation makes me quite happy that I use 1Password (an app on my computer) instead of a service.

My last point on this is that people need to upgrade if they’re using an affected version of OpenSSL. If you are, and your operating system provides it (which is the case with Red Hat Enterprise Linux and Fedora, among many others) then you really should be updating to the packages provided. It’s not a question of whether you should or shouldn’t — you should. Period. This has been a crazy week and a lot of crazy things have happened and this is a really really bad thing IF you’re affected. So if you are (as in you’re running Red Hat Enterprise Linux 6.5 or a current Fedora, etc.) then you really need to update ASAP. And then you need to assess your next steps (changing passwords on vulnerable (and now fixed) services, revoking and reissuing certificates if you feel it necessary, etc.).

Anyways, that’s all I have to say about Heartbleed. It will be interesting to see what the next few weeks will be like as we continue to get a bigger picture of what’s happened here, how, and to whom. And to see what damage has been done, and who responded appropriately and when. For instance, if there were a site or service I was using and as of today (being Saturday, and this thing exploded on Monday) it was still NOT fixed, I don’t think I would be using that site/service anymore. To put it into perspective, Red Hat had updates out late Monday for Red Hat Enterprise Linux 6.5 and the other affected products early Tuesday morning (my time). Everything was available to customers in under 24hrs. It’s not hard to install — “yum update” and reboot (to make sure everything is covered). So for a site to be still affected by this now? There’s really no excuse as far as I’m concerned.

Finally, just to note that I did get some minor press coverage (so this is more vanity than useful), LinuxInsider reported on Heartbleed and my name is noted, although my answers to the questions must have been less than exciting as there wasn’t too much noted there other than where Red Hat customers could go for more info. =)

And to finish off, the obligatory xkcd:

I’m FedUp with Fedora!

Sorry, couldn’t resist. =)

So normally I do my updates from one version of Fedora to the next using yum, in particular the Upgrading Fedora using yum guide. Usually it works pretty good. I didn’t really have much good experience with PreUpgrade the few times I tried it, so I wanted to give FedUp a try.

In my Parallels Fedora 17 VM it worked amazingly well. So decided to try it on my laptop, which is also running Fedora 17. I think it makes sense to do a little bit of house-keeping before running it though, and the FedUp page doesn’t mention any of this (perhaps it’s no longer needed?). Anyways, a few steps:

* yum install rpmconf; rpmconf -a (review any .rpmnew/.rpmsave files, merge changes as required)
* find /etc /var -name '*?.rpm?*' (find any other old .rpmnew/.rpmsave files)
* yum install yum-utils; package-cleanup --leaves (review and remove any unused packages, not all will be removable)
* package-cleanup --orphans (find and remove any orphan packages no longer in the repositories)

Now you can run FedUp:

* yum install fedup
* fedup-cli --network 18 --debuglog /root/fedupdebug.log

If this completes without error (check the log), you can reboot. At grub you’ll see a “System Upgrade” entry. When that’s done, it’ll reboot into Fedora 18.

The wiki page talks about upgrading GRUB2 since you’ll be booting from Fedora 17’s GRUB2. If you’ve got a BIOS-based system, you can use the Updating GRUB2 configuration on BIOS systems instructions. For those using UEFI, instructions are on the same page.

You may also want to run package-cleanup --orphans after you do the upgrade as well, just to get rid of any other leftovers. The only issue I discovered so far with the upgrade is that Google Chrome didn’t work out-of-the-box. However, doing a yum remove google-chrome-stable; yum install google-chrome got that sorted out (although it did install the unstable version; the stable version had issues with missing libraries and wouldn’t load).

All in all, upgrading from Fedora 17 to 18 went a heck of a lot smoother than a fresh install did. I also got to see what the new GDM/GNOME looks like (quite nice, actually, although I think I’ll give MATE a try on the laptop as well because, while GNOME3 is pretty, I definitely preferred GNOME2).

Good job, Fedora-folks! Now I just have to upgrade my main workstation, but I think I’m going to play on the laptop for a bit before taking that step. Just in case I find any other gotchya’s.

Systemd article on wiki

In light of my recent (mis-)adventures with Fedora 18 installation in VMware Fusion and the need to figure out some systemd stuff, I’ve started writing a hints/tip sheet type entry on my wiki: Systemd. Far from complete yet, but I’m going to use it to document some hints, tricks, light-bulb-moments, and comparisons to SysV-init tools (chkconfig, service). I’m no stranger to alternative boot/service management systems but it indeed makes me chuckle when people were complaining about Annvix using runit to manage the init system. I definitely appreciate the work gone into Systemd (having done a similar thing myself for Annvix although obviously not anywhere near the scale/scope of Systemd!), so do want to learn it as opposed to using old scripts as a crutch-interface to the new stuff. So as I likely stumble across various bits of useful info, I’ll be adding it to that wiki article if anyone else is interested in checking it out (or perhaps giving me tips… like how the heck do I actually get gettys to run in graphical.target!! I want a working CTRL-ALT-F2/F3 to login at the console please!!).

Upgrading and Installing Fedora 18 in VMware Fusion

I was thoroughly excited about Fedora 18 until I actually went to install it. I use Fedora 17 on a laptop and a desktop, but I also have copies in virtual machines that I use for testing — these virtual machines run in VMware Fusion on my Mac Pro. So never having used Fedora 18 during any of the betas or anything, first thing I did was try to install it in VMware.

A few times.

First I have to say that I’m not a big fan of the new Anaconda UI. One of the things I appreciated about Anaconda in the past is that it was easy to use, but didn’t prevent me from doing “power things”, like partitioning disks the way I wanted to partition them. This seems to no longer be the case, and Anaconda does a lot of hand-holding. It has been a habit to go through the list of packages to be installed (select a group, then go through and weed out the stuff I don’t want), so things I don’t want don’t even touch the disk. Doesn’t seem like I can do this anymore either. I don’t agree with some reviewers who seem to think the UI change is to make it look/feel more mobile or “touchscreen-like”, but I do feel it’s dumbed down quite a bit which makes me unhappy. I lived with it, until trying to change the partition layout killed Anaconda. Twice. After that I decided to let it do what it wanted since it was just a VM and I didn’t care enough to fight it.

So Fedora gets installed and lo! A nice blue screen where I can do absolutely nothing. No icons, no text, no login box, nothing. Afterwards I read this has to do with having 3D graphics enabled in VMware (but at that point I had already reinstalled to use LXDE rather than GNOME, so I’ve not tried anything regarding that, however I did find this blog posting about Fedora 18 in VMware Fusion 5/VMware Workstation 9 which explains things a bit).

On my home network I run IPA on Red Hat Enterprise Linux 6 to handle my Kerberos/LDAP auth duties. This means when I install an operating system, no matter what it is, I don’t create local users. So when I installed Fedora 18, I only have the root user and the rest come from IPA. A few problems with that:

  • Setting up IPA in Anaconda does not work
  • Post-install, logging in as root at the GUI is not permitted
  • I have a grub error on tty2, not a login (and nothing on tty3 for that matter)
  • I have to reboot and edit grub to add systemd.unit=multi-user.target to boot into the console to get a tty (see Systemd: Boot Kernel Command Line)
  • After I install freeipa-client, I can enroll in the domain
  • SSSD is not enabled when you install it! Once again I have to reboot to the console
  • Enable SSSD and reboot, only to find that once again I cannot login as a user in the GUI (that I could login as on the console)
  • Head-scratching moment when I realize I know next to nothing about systemd and can only conclude that perhaps NetworkManager starts upon login in the graphical boot and not at system start, which prevents SSSD from talking to IPA to confirm my login
  • Looking at /etc/sysconfig/network-scripts/ifup-eth0 tells me that perhaps that is not the case
  • Big WTF moment and go to bed

Currently I am unhappy with Fedora 18. I suppose that on bare metal it probably works better. I also suppose that with a real user account it works better. At this point I don’t want to do yet another reinstall because then I have to remove the host from IPA and re-enroll, which is annoying (and I really don’t want to revisit Anaconda either — I’ll get to do that when I create the 64bit Fedora 18 VM later).

Ok, after a few days I tried a few other things. The first was getting rid of NetworkManager thinking that perhaps it would only start when I login (which I can’t, if there is no network, because my login is IPA-based). So I had to do:

# systemctl disable NetworkManager.service
# systemctl enable network.service

The most annoying thing, however, is that when I boot into graphical mode, I get no gettys. I’m pretty sure I told systemd that I want three of them (I have three symlinks in /etc/systemd/system/getty.target.wants: getty@tty1.service through getty@tty3.service; sadly when I boot into graphical mode and switch to TTY2, I get a grub error message and nothing on TTY3… what the heck?)

Another thing I did was disable SELinux. In multi-user.target, I can login as my IPA-user no problem. With LXDM, however, I never could (assuming, perhaps incorrectly, that network initialization was delayed). But upon further inspection I see a lot of:

lxdm-binary: pam_selinux(lxdm:session): Setting key creation contect guest_u:guest_r:oddjob_mkhomedir_t:s0 failed: Permission denied

I have zero interest and zero time in making SELinux work with LXDM and since this is a virtual machine, I don’t care if SELinux is disabled.

Lo and behold, I can now log into LXDM. In light of the SELinux thing, I’m thinking that NetworkManager is not to blame (silly me, I made two changes then rebooted — it could be either one, but I strongly suspect SELinux). Maybe it would have worked better with GNOME (will try that on the next VM I install).

On another side note, since I use VMWare Fusion for the VM, for the tools install (probably the same for the latest VMware Workstation as well, etc.), you may want to check out this thread. In particular, seems like linux/version.h is missing and makes the tools angry. The fix is easy though:

# cd /usr/src/kernels/[current_version]/include
# cp generated/uapi/linux/version.h linux/

Then you can compile the tools. IIRC, you’ll also not want to use the i686 PAE kernel (doesn’t seem to work on older versions, I suspect the same may be true with F18). Easy enough to yum remove the PAE kernels and install the regular kernel. YMMV.

Finally, the next day I installed Fedora 18 64-bit in VMWare Fusion. With the lessons learned above, it was a lot less painful. I still am not a big fan of Anaconda… I like the look of it, but the practical “working” bits of it definitely need some work. I don’t like that you can pick only one group of packages. What if I want both GNOME desktop and a working web server? Doesn’t seem like it can be done. I also like picking through individual packages and sorely miss that — makes for having to install a bunch of stuff later. Also not sure why vim isn’t installed by default… I think this is the first time ever I’ve seen “vim: command not found”.

Also, despite getting the VMware tools installed, if you want GDM to present you with anything, you need to disable 3D graphics. I have it enabled on my 32-bit VM (which is using LXDM, works fine), but on the 64-bit VM (using GDM), even with vmware-tools compiled and running, I needed to disable 3D graphic acceleration. I suppose there is a bug in the driver and/or X there. Not a big deal to disable since I’ve done “yum groupinstall ‘MATE Desktop'” (what need do I have for 3D graphics? BTW, MATE is sweet!) and now GDM loads.

IPA still would not work to configure from Anaconda. Definitely a bug to be filed there.

Anyways, time to end this. This entry has been in the works for about two weeks (sadly, I’ve not had time to finish it or get my Fedora VM’s running until yesterday, and finished up the 64-bit install this morning). So far the install of Fedora 18, in VMware, has been a bit of a PITA. Next order of business is to see how the upgrade works… I’ve got a Fedora 17 install in Parallels to upgrade to make sure all my work-related bits work, before attempting to upgrade on bare metal. I’m quite certain that a lot of this can be attributed to the virtual machines and it’ll work better on bare metal. I’m also happy that those won’t be fresh installs, so I don’t have to deal with Anaconda.