15:00:17 #startmeeting oVirt Infra weekly 15:00:17 Meeting started Mon Nov 26 15:00:17 2012 UTC. The chair is quaid. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:17 Useful Commands: #action #agreed #help #info #idea #link #topic. 15:00:19 dneary: Thank you. I have a support question I'll ask it in an hour or so then. Thank you. 15:00:25 #chair dneary 15:00:25 Current chairs: dneary quaid 15:00:34 adrian15, Ask away 15:00:46 adrian15, Some people may be able to help you 15:00:47 * eedri here 15:01:09 * garrett_ is here 15:02:24 #topic Agenda 15:02:43 ''Agenda'' 15:02:43 * MediaWiki & www 15:02:43 * MediaWiki backup script 15:02:43 * Hosting 15:02:43 * Trac review 15:02:45 * Puppet 15:02:48 * Jenkins 15:02:50 * Gerrit 15:02:53 * Other business? 15:02:55 Anything to add to that agenda? 15:03:13 process of adding new admins to infra? 15:03:21 ok 15:03:24 i want to introduce a new candidate 15:04:08 ok, let's move on then 15:04:16 #topic MediaWiki & www 15:05:03 * sgordon is belatedly here 15:05:04 #info Remaining problem of a mod_rewrite rule to get the logo visible 15:05:19 does anybody know mod_rewrite enough to help me troubleshoot this? 15:05:36 I have some knowledge, I can try 15:06:19 dcaro: great, thanks can I show you what's up right after the meeting? 15:07:39 dcaro: actually, here it is: 15:07:42 http://fpaste.org/NEgP/ 15:07:44 that's the .htaccess 15:07:55 quaid: (it does work on my instance too fwiw, so we can get it working somehow) 15:08:04 and the problem is this: 15:08:06 http://wiki-ovirt.rhcloud.com/skins/common/images/oVirt-logo.png 15:08:34 that gets read as an article, so the s/skins/Skins/ is changed (and then not found) 15:08:44 garrett_: right, thanks for reminding of that :) 15:08:52 quaid: it might be in the LocalSettings.php file too 15:09:06 not sure if Apache or MW handles that 15:09:10 (for the logo) 15:09:38 Apache should be finding it as a file 15:11:10 that's all I know of that is not working 15:11:20 ... but I haven't gone looking through every page for images, etc. :) 15:11:20 quaid: Error: ".." is not a valid command. 15:11:27 I'll look again later, use the fresh perspective thin 15:11:36 whoops, mis-paste on ^^^^^ 15:12:57 #info process to transfer sites should take 45 to 90 minutes 15:13:18 #action quaid to arrange DNS cutover timing based on the idea that we'll have this working within 2 days 15:13:37 anything more on this topic? 15:13:57 we can work on fixing it here in channel after the meeting/as the day progresses 15:14:18 quaid, I'm thinking those duplicate RewriteConds are redundant... 15:14:28 quaid, Let me dig around, make a suggestion 15:15:28 dneary: I added the fourth rule block 15:15:41 the first three are what garrett_ passed over that should work 15:15:43 quaid, dneary:I think that the first one should be the last... 15:16:12 dcaro, Yes, I think that's the issue 15:16:44 quaid, Apache's processing the rewrite rules from top to bottom, and MediaWiki is taking care of all URLs since the redirect to index.php gets added first 15:17:21 anyone know of any pages that have images on them to see if the regular images are showing up? 15:18:07 dneary: dcaro ah but ... 15:18:24 I thought that the first two rule conditions say, "if it's not already a folder or file, then redirect to index.php" 15:18:30 and that image is a file in a folder 15:18:44 the rule there works for e.g. robots.txt 15:19:01 ok, sorry, I don't want to get caught up here in fixing this 15:19:02 we 15:19:21 don't stop if you want to keep looking, but I'll move the agenda along now 15:19:31 quaid, Yes, you're right, I think 15:19:41 So the same rules are not applying to skins/... 15:19:45 #info need to make sure wiki images are showing up in pages 15:19:53 (also preceded with -f -d 15:19:56 #topic MediaWiki backup script 15:20:32 #info I forgot to remove this topic but I think we're covered with this because we'll need a new backup procedure for the new OpenShift based tool 15:20:48 #info of course if I write that script it may have the same problem, so I'll be coming back later for help on that :) 15:20:56 #topic Hosting 15:21:23 #info My Red Hat IT contact told me last week he is provisioning two servers at RackSpace for us, bare metal to use as we wish 15:21:44 #info RackSpace servers should be available this week or next, depending on how fast the paper churns 15:23:24 quaid, The back-up script, as annoying as a daily email is, is low priority 15:23:28 #info We're also proceeding with plans for Alter Way work, and may start installing & such but not switchover data until paper churns there 15:23:34 We should just let people know that on the list 15:23:44 oh, ok 15:23:56 is the problem the broken part or the daily email? 15:24:08 quaid, re Alter Way, it remains to be seen if we'll need paper at all :-) 15:24:15 quaid, The daily email 15:24:19 I'd considered having it send "I'm done" email, but maybe it could just trigger the logwatch email instead 15:24:30 quaid, Ppl are asking just to have the email stopped 15:24:47 I seem not to have seen this thread, I'll look and reply 15:24:54 Personally I don't care that much. One email more or less very day... 15:25:06 #action quaid to tell people status of 'broken' script & to please hang on 15:25:23 ok, moving along .. 15:25:30 quaid, It's come up several times - typically when someone replies to the "mediawiki backup script" error messages 15:25:33 dneary: if you are done with hosting? 15:25:37 Done 15:25:51 No news to report beyond mailing list & last week's meetings 15:27:01 word 15:27:15 #topic Trac review 15:27:43 #info I'm moving this topic to the end of the meeting, as it's a bit of administrivia that we don't need to do in the middle - often it's covered by other open items anyway, so putting it at the end will make it faster 15:27:55 #action quaid to move this topic to the end of the agenda permanently 15:27:58 make sense? 15:28:03 +1 15:28:20 cool 15:28:30 #topic Puppet 15:28:42 if ewoud isn't available, I can update on this one 15:30:45 #info ewoud has a proof of concept (POC) Foreman install running: 15:30:57 https://foreman.ekohl.nl 15:31:25 #info ewoud is working on the initial Puppet manifests 15:31:51 hmm, I forget who else volunteered to help learn & write those? was that dneary ? 15:32:04 quaid, I did 15:32:10 Haven't had any time for that this week 15:32:57 I can help too, I have some experience with foreman-puppet 15:33:39 no worries 15:33:44 dcaro: +1 15:34:02 i can say dcaro has experience in jenkins,puppet,foreman 15:34:06 #action quaid, dcaro, dneary will help ewoud with Puppet manifests 15:34:19 he's part of the team that manage those services in rhevm 15:34:28 so +1 15:34:51 #info Puppet manifest discussion to happen on infra@ovirt.org ASAP 15:34:54 great 15:35:06 yep :), proud of it 15:35:40 dcaro: we have two new hosting situations starting with 2 servers at each facility, and we're going to do all the provisioning for them using Foreman/Puppet - so nothing in the new infrastructure is hand-built :) 15:36:00 we can discuss the rest of the details on list - what is going to be where, etc. 15:36:20 sure 15:36:26 dcaro, current evironemnt mainly consists of amazon ec2 vms 15:36:36 dcaro, running all our services (gerrit/jenkins/wiki/etc..) 15:36:39 & linode01 :) 15:36:43 yap.. 15:36:48 the kitchen sink 15:36:53 wiki is on linode (with mailman) = the kitchen sink :) 15:37:06 rest is on EC2 15:37:24 but soon - all KVM, all the time! 15:38:06 * quaid ready to move on? 15:38:33 yes 15:38:57 yes 15:39:54 #topic Jenkins 15:40:36 jenkins.ovirt.org was upgraded during the weekend to latest LTS version 15:40:50 this was important due to latest security issues 15:41:06 #info jenkins.ovirt.org updated to latest for security fix 15:41:14 eedri++ 15:41:36 also, we have some new power users 15:41:58 recently we started delegating powerusers to handle specific jobs on jenkins.ovirt.org 15:42:30 this reduce load on the infra team and gives an oppertunity to users with experience and will to enhance ovirt sub projects in jenkins 15:43:04 #action we should consider using openid auth in new jenkins server 15:43:18 #info we've been adding new power users, who offest the load on the Infra tream by enabling people with experience to enhance sub projects in Jenkins 15:43:28 * ewoud here now 15:43:38 sorry, meeting ran a bit late 15:43:40 #idea we should consider using openid auth in new Jenkins server 15:43:57 eedri: do you want to work on that idea/proposal/activity? 15:44:00 * Rydekull is on a train right now, and about to change to a bus, so he'll be cut off, but wonders if there is something that the infra team needs help with 15:44:23 ewoud: no worries, look back on what we wrote above, we can go back to PUppet after this if you have more to add 15:44:39 quaid, yes, we'll test it once the new server is ready 15:44:46 quaid, shouldn't be too complicated 15:45:15 Rydekull: definitely, we can use help across the board - we'll be talking about how to bring in new people in a few minutes, but if you want to come chat with me here later when you're back connected, I can fill you in 15:45:32 #action eedri will work on OpenID for Jenkins 15:45:45 eedri: sounds like a great idea 15:45:58 quaid, i asked theforeman guys, the don't support it yet 15:46:11 I wonder if OpenID can be used with MediaWiki - it will still want to associate with a user account, but maybe it can create and keep the association or something 15:46:13 quaid, but they promised that if we open a feature request, they will :) 15:46:30 #action oVirt Infra to make a Foreman feature request to have OpenID 15:46:39 #action quaid to look in to MediaWiki and OpenID 15:46:51 anything more on Jenkins? 15:47:00 ewoud: do you want to return to Puppet/Foreman for a few minutes? 15:47:11 quaid: please 15:47:12 ewoud, +1 15:48:26 btw, new jenkins will have both fedora 17 & 18 slaves? 15:48:27 change the topic for the minutes? 15:48:47 #topic Puppet/Foreman 15:49:03 so last weekend I set up a PoC foreman instance on http://foreman.ekohl.nl 15:49:27 I've been keeping some documentation on https://github.com/ekohl/foreman-howto so it should be easy to reproduce 15:49:40 ewoud: but i dont have permissions to play with your instance ;) 15:50:00 ohadlevy: I can make an account 15:50:18 ewoud: lol, I got enough foreman instances ;) 15:50:20 so can eedri 15:50:22 ohadlevy: :) 15:50:49 I've also written some puppet classes for my jenkins slave which are available on https://github.com/ekohl/ovirt-infra-puppet 15:51:30 I'm currently working on making it easy to install new hosts so you can place with the provisioning, but haven't had time to finish that 15:52:09 sadly there's no foreman - ganeti integration, so there has to be some manual work 15:52:18 but it's a PoC ;) 15:52:37 eedri: well,, patches are welcomed, I'm sure fog users would appreciate it 15:53:04 ohadlevy: I should learn ruby, but first I'm going to focus on the foreman-installer patches 15:53:50 ewoud: did you see above that dcaro has experience with Foreman & Puppet and is interesting in helping, and dneary and I don't have experience and will be learning-while-helping? :) 15:54:23 quaid: yes 15:54:26 great 15:54:47 ok, anything more on foreman/puppet? 15:54:50 but now we have machines coming we can focus on what we want to automate 15:55:15 we've already said jenkins and gerrit I think 15:55:40 maybe try to work those out? also a bit of base system such as how do we manage accounts 15:56:22 those are things you can also think about without puppet knowledge :) 15:56:41 quaid, we're going to have seperate git repos? 15:56:49 quaid, one for puppet and one for forman? 15:57:16 I think ewoud said yes? 15:57:41 not sure what you want in the foreman repo 15:58:30 ewoud, if we're using the rpms, then i guess we don't need 15:58:50 I'm strongly in favor of RPMs 15:59:00 ewoud, i agree 15:59:03 ok, I want to see if we can hit the last topics without going much over an hour 15:59:08 ewoud, as long as foreman supports upgrades 15:59:30 #agreed it's better to package scripts and stuff in RPMs than to have a git repo for scripts < Foreman 15:59:34 is that a correct agreed? 16:00:12 quaid, it's better to say that using foreman rpms deployments is better than using the foreman git repo 16:00:22 yes, and any system specific script we can deploy using puppet with files/templates if needed 16:00:29 when you runonce a VM with certain parameters and reboot, i noticed that VM still uses those given parameters...isnt that a bug...? 16:00:51 karimb: the run means run in terms of the KVM process living 16:01:14 karimb: so a guest reboot doens't count because KVM stays alive, you really need to shut down and boot again to lose run once 16:01:29 #undo 16:01:29 Removing item from minutes: 16:01:30 eedri, ewoud, quaid: I agree, rpms better than repos 16:01:30 ewoud, so is there a way to prevent this behaviour? 16:01:53 #agreed using foreman rpms deployments is better than using the foreman git repo 16:01:59 ewoud, cos it makes useless booting from kernel+initrd+cmdline and forget about VM s installation.... 16:02:06 #topic Gerrit 16:02:12 karimb: not afaik, but been a while since I really looked at it 16:02:13 anything on this topic? 16:02:26 #info I think the Infra git repo is available, but I haven't put anything in it 16:03:50 moving on ... 16:04:15 #topic Process for bringing in new people to INfra 16:04:21 ewoud, allright ...will use poweroff in my kickstart and a script running on manager to boot VMS tagged with first_install 16:04:32 #info we need a discussion on list where everyone can be involved 16:04:51 #action quaid to start new thread ASAP 16:05:49 we'll just discuss in that thread, get it figured out in the next day or so 16:05:59 as we have a few people right now in that position 16:06:02 #topic Trac review 16:06:11 https://fedorahosted.org/ovirt/report/1 16:06:27 I should update my foreman/puppet ticket 16:06:32 great 16:06:42 I'll just run through the tickets we didn't discuss otherside in this meeting 16:06:47 https://fedorahosted.org/ovirt/ticket/10 16:07:16 #info renaming of mailing list arch > devel is happening after the www/wiki transfer is all worked out, priority-wise 16:07:37 #action quaid needs to put together a migration plan, so people are warned to fix mail filter rules, etc. 16:07:55 https://fedorahosted.org/ovirt/ticket/11 16:08:11 #info slow mailing list show resolve when we move it to Alter Way, leaving this open for now so we can be sure it is better 16:08:46 https://fedorahosted.org/ovirt/ticket/13 16:09:06 #info Is this an Infra task? 16:09:15 #action quaid to find out if this is an Infra ticket or now 16:09:43 https://fedorahosted.org/ovirt/ticket/5 16:10:09 priority wise this has fallen below puppet/foreman 16:10:18 #info We'll need to have a how to file a ticket process in place one before doing all the hosting switching 16:10:34 ewoud: I can takeover this ticket, since my big hard work has mostly completed :) 16:10:46 quaid: feel free to take it 16:10:53 #action quaid to takeover this ticket 16:11:07 besides, docs is what I know how to do the best anyway :) 16:11:20 https://fedorahosted.org/ovirt/ticket/12 16:11:59 #action quaid to revive the thread on this, get it fixed or lined up to change with new hosting 16:13:05 ok, that's it 16:13:11 closing meeting in 30 seconds 16:14:21 adrian15, You're up :) 16:14:54 #endmeeting