minutes for the LibreOffice QA call 2012-04-06 1400UTC

[Bjoern_Michaelsen1]

Hi all,

here are the minutes for the LibreOffice QA call on:

Friday, 2012-04-06 1400UTC

attending: Rainer Bielefeld, Florian Reisinger, Petr Mladek, Jan Holesovsky
(Kendy), Bjoern Michaelsen

* pending action items
  - Update/Create active triagers wiki page (Cor/Rainer)
  - Publish Rainers charts'n data on blog/planets (Cor)
AA poke infra to add Rainers Blog to planet (Bjoern)
  - collect further ideas for spending a dedicated resource (Cor/Rainer)
    -> blocked: to get a concise list need to make a CaseConductor/Litmus
                choice

  - Set Cor up with the Community/Forum maintainers at the distros
    to better propagate Hackfests, Bug Hunting Sessions etc.
    RedHat, Debian, Gentoo still missing

  - Walkthrough setup at Hackfest to find out what need better docs
    (Bjoern/Rainer)
  - Setting up a ready-to-go VirtualBox with everything installed would be
    cool (Bjoern/Korrawit?)
    https://bugs.freedesktop.org/show_bug.cgi?id=47648
    
  - automated test docs: really straightforward for Calc, just needs more
    CSV test documents (all)
    https://bugs.freedesktop.org/show_bug.cgi?id=47667
    Hackfest

* completed action items
  - Check if test documents are URLs properly distributed to Checkbox (Bjoern)
    URLs are there, but they are not hyperlinked
  - Set Cor up with the Community/Forum maintainers at the distros
    Ubuntu: done, SUSE: done (Bjoern, Petr)

* structured manual testing
  - tests are in checkbox
  - Call for Testing is out:
    http://www.theorangenotebook.com/2012/04/opportunity-manual-application-testing.html
AA Blog about this refering to checkbox for Ubuntu and Litmus for others (Bjoern)
  - CaseConductor evaluation
    - Sophie Gaultier did some great research:
      - CaseConductor now called MozTrap, to raise confusion
      - would need more permissions/roles to investigate in Mozillas staging instance
    - Petr also had a look:
      - a bit confusing at first
      - but a lot nicer/modern than Litmus
      - might be easier if setup properly
    - our own instance probably the way to go as we need an admin for it in the
      end anyway
    - forward to infra to get a staging instance setup (Sophie/Bjoern)
  - Rainer: we need better documentation on the "why" of Litmus/MozTrap
    - wiki pages contain a lot of detailed information about the how, but
      little on the why
    - important topic to get started on the Hackfest
  - Kendy: Need for regular manual testing for update scenarios
    - in theory classical example case for Litmus/MozTrap
    - also still for 3.5 release series
AA - write testcase in Litmus (Kendy)
    - given the limited current reach of Litmus, we might propagate in blogs
      too
  - we really need to make structured manual a lot more visible for 3.6

* bugwrangling
   - bugwrangling documentation (Florian)
     - current bug wrangling docs are to complex (Florian)
     - wiki docs are ad-hoc and it shows (Rainer)
AA - good old plain document for bugwrangling beginners is in the works (Rainer/Florian)
AA - discuss this along with other wiki cleanup at the Hackfest (Rainer/Bjoern)
     - get the documentation team involved maybe? (Rainer)
     - in the wiki rework also consider entry points from ask.libreoffice.org (Bjoern)
   - generic bug tagging
     - currently not a pressing issue (Rainer)
     - in general whiteboard status is good for this (Petr)
     - tagging in summary might cause trouble (Bjoern)
       (::rtl::OUString bugs vs. Right-To-Left bugs for example)
     - agreement that the 50 current RTL bugs are not that problematic yet
     - for the future we should prevent summary tagging to become a custom
     - also excessive summary tags cause trouble in bugmail (Petr)
AA - bulk change remove EasyHack from summary, make that whiteboard tagging only (Rainer)
   - Florian setup a VirtualBox image for Linux regression testing
     - Ubuntu precise/12.04
     - has LibreOffice 3.5.x and LibreOffice 3.4.x with Rainer parallel
       installation script
AA - provide Florian with a LibreOffice 3.3 build for this
     - publishing VirtualBox somewere for broader audience (Bjoern)
AA - needs webspace (Bjoern)
     - Florian cannot be at the Hackfest
AA - meet up remotely (IRC/Skype), find time/date (Rainer/Bjoern)

* automated testing
   - Regina Herschel and Markus Mohnhard will be at the Hackfest, looking
     forward for good progress there

* bibisect for 3.5 release branch and 3.6 master
AA - provide 3.5 release and 3.6 master bibisect updates

* next call:

  Friday, 2012-04-20 14:00UTC

Corrections and additions most welcome. I am most happily surprised by Florian
showing up and also having an interesting project at hand!

See most of you at the Hackfest in Hamburg!

Best,

Bjoern

[Petr_Mladek]

Bjoern Michaelsen píše v Pá 06. 04. 2012 v 19:55 +0200:

* structured manual testing
  - tests are in checkbox
  - Call for Testing is out:
    http://www.theorangenotebook.com/2012/04/opportunity-manual-application-testing.html
AA Blog about this refering to checkbox for Ubuntu and Litmus for others (Bjoern)
  - CaseConductor evaluation
    - Sophie Gaultier did some great research:
      - CaseConductor now called MozTrap, to raise confusion
      - would need more permissions/roles to investigate in Mozillas staging instance
    - Petr also had a look:
      - a bit confusing at first
      - but a lot nicer/modern than Litmus
      - might be easier if setup properly
    - our own instance probably the way to go as we need an admin for it in the
      end anyway
    - forward to infra to get a staging instance setup (Sophie/Bjoern)
  - Rainer: we need better documentation on the "why" of Litmus/MozTrap
    - wiki pages contain a lot of detailed information about the how, but
      little on the why

I wonder if people know the pages:
http://wiki.documentfoundation.org/QA/Testing/Test_Case
http://wiki.documentfoundation.org/QA/Testing/Regression_Tests
http://wiki.documentfoundation.org/QA/Testing/Feature_Tests

They provide also some "why" information and are linked from
http://wiki.documentfoundation.org/QA . Of course, they are not perfect
but I have the feeling that people do not know them at all.

Hmm, normal tester would start with
http://www.libreoffice.org/get-involved/qa-testers/ and this page is
outdated.

Florian, could you please replace:

--- cut ---
* Manual testing: you can perform manual tests on the development and
release candidate [RC] versions of LibreOffice. This process is
currently under development and will be soon available under a Test Case
Management System. In the mean time, you can get in contact with your
Language Community to contribute to the process used for the 3.3
release.

  * Test reporting: The manual testing wiki page contains tables for
    reporting on tests (you'll need to create a wiki user account or log
    in beforehand). If you identify a problem when you're testing,
    report it on our developers mailing list (see our global mailing
    list index) and, if the bug has not already been reported on the bug
    tracker, file a bug report yourself.
--- cut ---

with something like:

--- cut ---
Manual Testing: you can help with regular regression tests[1]. Just
choose a test cases in Litmus[2] and follow instructions. If you miss an
useful check, go and add it[3]. Finally, you could help stabilizing new
features[4]. Just take one, play with it, and report problems.

Automatic Testing: It might be an interesting introduction into the
LibreOffice development if you write your own automatic check[5]. Even
non developers could provide test documents for some special tests[6].
--- cut ---

Reference:

[1] http://wiki.documentfoundation.org/QA/Testing/Regression_Tests
[2] https://wiki.documentfoundation.org/Litmus
[3] https://wiki.documentfoundation.org/QA/Testing/Test_Case
[4] http://wiki.documentfoundation.org/QA/Testing/Feature_Tests
[5] http://wiki.documentfoundation.org/QA/Testing/Automated_Tests
[6]
http://wiki.documentfoundation.org/Development/Unit_Tests_By_Non_Developers

Best Regards,
Petr

[cornouws]

Hi,

Bjoern Michaelsen wrote (06-04-12 19:55)

   - CaseConductor evaluation
     [...]
   - Rainer: we need better documentation on the "why" of Litmus/MozTrap
     [...]
   - Kendy: Need for regular manual testing for update scenarios
     [...]
     - given the limited current reach of Litmus, we might propagate in blogs
       too
   - we really need to make structured manual a lot more visible for 3.6

I look forward to the outcome of the Litmus/MozTrap evaluation and improvements in this area. It will help people run the tests more easily. However, we must not forget that it's just that and not more.
Evaluation of the current situation showed some lack of available time with people considered to work with it (and other QA topics). Thus if we forget that, and only do improve the tooling... it might turn out to be fooling our selves.

Cheers,

[sophi]

Hi Cor,

Hi,

Bjoern Michaelsen wrote (06-04-12 19:55)

- CaseConductor evaluation
[...]
- Rainer: we need better documentation on the "why" of Litmus/MozTrap
[...]
- Kendy: Need for regular manual testing for update scenarios
[...]
- given the limited current reach of Litmus, we might propagate in blogs
too
- we really need to make structured manual a lot more visible for 3.6

I look forward to the outcome of the Litmus/MozTrap evaluation and
improvements in this area. It will help people run the tests more
easily. However, we must not forget that it's just that and not more.

If it's easy, it will attract people too. QA is frightening for a lot of users because it looks like technical when it's not. The tool should help that and currently Litmus is not fulfilling that.

Evaluation of the current situation showed some lack of available time
with people considered to work with it (and other QA topics). Thus if we
forget that, and only do improve the tooling... it might turn out to be
fooling our selves.

In that case we know that we can't extend people time, so let extend the number of contributors by providing an easy way to participate. Tools are only tools but this is what our contributors are playing with, see Pootle for example, it's the same here, it should be as easy as and as flexible as Pootle.
Kind regards
Sophie

[cornouws]

Hi Sophie,

Sophie Gautier wrote (10-04-12 23:05)

If it's easy, it will attract people too. QA is frightening for a lot of
users because it looks like technical when it's not.

Ah well, of course it is technical compared to what many people are used to, but I agree with you that easier tools lower the barrier and thus help