Difference: QualityAssurance (1 vs. 5)

Revision 521 Nov 2007 - Main.BobMorris

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

QualityAssurance

Line: 10 to 10
 
Changed:
<
<		[Jim's diagram in this document doesn't quite coincide with his outline(s) below the diagram. In Test Driven Development methodologies such as eXtreme Programming, but if this is to be part of the documentation, the TDWG audience might be better served with closer association between the examples and the diagram. -- BobMorris - 20 Nov 2007.]
>
>		[Jim's diagram in this document doesn't quite coincide with his outline(s) below the diagram. In Test Driven Development methodologies such as eXtreme Programming, the tests drive the design or are parallel to it, but in more traditional methodologies, tests are derived from the design, and this seems to be what his examples follow. If this is to be part of the documentation, the TDWG audience might be better served with closer association between the examples and the diagram. -- BobMorris - 20 Nov 2007.]

[What Quality are we Assuring and for What? Maybe I'm missing some definitions somewhere? I think there is quite a difference between assurance of quality during a software development lifecycle, and assuring quality of "standard" software or of a standard. When does TDWG enter the scene? Before, during or after the process of development of software or standards? Some of the questions listed here are quality tests, that ask have the proper steps been followed after the fact rather than the processes of building in quality during the lifecycle that QA implies. Quality tests for standards would be slightly different than quality tests for software, so I think you need to divide them. So, maybe this whole topic area should be called Quality Testing or Validation -- ChuckMiller - 20 Nov 2007]

[Main.ChuckMiller makes an excellent point. Sometime in the past, StanleyBlum and others looked at the procedures for IETF standards adoption described in RFC2026 and those for adoption by W3. Both of those documents reflect well the differences between characterizing the maturity level of standards and that of software. I wonder if they are more relevant, and what ever happened to the examinations of the procedures of those two organizations. -- BobMorris - 21 Nov 2007]

 Discussion is needed as to which elements from this lifecycle should be enforced in TDWG processes.
Line: 26 to 31
 
  • Alternatives Analysis - What alternative approaches have been considered and why are they ruled out?
[Although not exactly the same, some writers call this Rationale Documentation. That is meant to refer to discussion of what choices were made when there are several alternatives meeting the requirements. In such a document, one would never see a discussion of an alternative excluded because it did not meet the requirement. Rationale documents are the savior of many a maintainer with no access to the original developer's thoughts, because they can signal unavoidable dependencies or other pitfalls that may result from re-engineering in ways that don't follow the rationale. Maybe for standards, both alternatives and rationales are important. Not sure if they belong in a single rubric or not, but if so they need clear separation. -- BobMorris - 20 Nov 2007]
Deleted:
<
<		[What Quality are we Assuring and for What? Maybe I'm missing some definitions somewhere? I think there is quite a difference between assurance of quality during a software development lifecycle, and assuring quality of "standard" software or of a standard. When does TDWG enter the scene? Before, during or after the process of development of software or standards? Some of the questions listed here are quality tests, that ask have the proper steps been followed after the fact rather than the processes of building in quality during the lifecycle that QA implies. Quality tests for standards would be slightly different than quality tests for software, so I think you need to divide them. So, maybe this whole topic area should be called Quality Testing or Validation -- ChuckMiller - 20 Nov 2007]
 

  • Test Plan - How will we know if the product meets its objectives? How will we know if it has stopped meeting its objectives?

Revision 420 Nov 2007 - Main.ChuckMiller

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

QualityAssurance

Line: 26 to 26
 
  • Alternatives Analysis - What alternative approaches have been considered and why are they ruled out?
[Although not exactly the same, some writers call this Rationale Documentation. That is meant to refer to discussion of what choices were made when there are several alternatives meeting the requirements. In such a document, one would never see a discussion of an alternative excluded because it did not meet the requirement. Rationale documents are the savior of many a maintainer with no access to the original developer's thoughts, because they can signal unavoidable dependencies or other pitfalls that may result from re-engineering in ways that don't follow the rationale. Maybe for standards, both alternatives and rationales are important. Not sure if they belong in a single rubric or not, but if so they need clear separation. -- BobMorris - 20 Nov 2007]
Added:
>
>		[What Quality are we Assuring and for What? Maybe I'm missing some definitions somewhere? I think there is quite a difference between assurance of quality during a software development lifecycle, and assuring quality of "standard" software or of a standard. When does TDWG enter the scene? Before, during or after the process of development of software or standards? Some of the questions listed here are quality tests, that ask have the proper steps been followed after the fact rather than the processes of building in quality during the lifecycle that QA implies. Quality tests for standards would be slightly different than quality tests for software, so I think you need to divide them. So, maybe this whole topic area should be called Quality Testing or Validation -- ChuckMiller - 20 Nov 2007]

 
  • Test Plan - How will we know if the product meets its objectives? How will we know if it has stopped meeting its objectives?
  • Maintenance Plan - How will the product survive in the longer term? How will defects be handled? How will users report them?
  • Documentation - What documentation will be produced to support the product?

Revision 320 Nov 2007 - Main.BobMorris

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

QualityAssurance

Line: 10 to 10
 
Added:
>
>		[Jim's diagram in this document doesn't quite coincide with his outline(s) below the diagram. In Test Driven Development methodologies such as eXtreme Programming, but if this is to be part of the documentation, the TDWG audience might be better served with closer association between the examples and the diagram. -- BobMorris - 20 Nov 2007.]
 Discussion is needed as to which elements from this lifecycle should be enforced in TDWG processes.

Enforcing Quality Assurance Processes

Line: 22 to 24
 
  • Use cases & Scenarios - Is it clear how the the intended audience will use this product? What is it for?
  • Feasibility Studies - What evidence is there this is a feasible thing to do? Even if this is a feasibility study there should be some evidence suggesting there is a potential benefit.
  • Alternatives Analysis - What alternative approaches have been considered and why are they ruled out?
Added:
>
>		[Although not exactly the same, some writers call this Rationale Documentation. That is meant to refer to discussion of what choices were made when there are several alternatives meeting the requirements. In such a document, one would never see a discussion of an alternative excluded because it did not meet the requirement. Rationale documents are the savior of many a maintainer with no access to the original developer's thoughts, because they can signal unavoidable dependencies or other pitfalls that may result from re-engineering in ways that don't follow the rationale. Maybe for standards, both alternatives and rationales are important. Not sure if they belong in a single rubric or not, but if so they need clear separation. -- BobMorris - 20 Nov 2007]
 
  • Test Plan - How will we know if the product meets its objectives? How will we know if it has stopped meeting its objectives?
  • Maintenance Plan - How will the product survive in the longer term? How will defects be handled? How will users report them?
  • Documentation - What documentation will be produced to support the product?
  • Training - What training is required?
  • Integration - How does this product integrate with other TDWG products and those of other groups?
Added:
>
>		[As to standards, where does "Implementation Plan" fit in all this. For example, IETF requires two independent implementations of a standard before final adoption. Does TDWG require any? If so, is something needed in this list? -- BobMorris - 20 Nov 2007]
 

Testing Related Projects

TAPIR Testing

Revision 205 Nov 2007 - Main.RenatoDeGiovanni

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

QualityAssurance

Line: 32 to 32
 

TAPIR Testing

Added:
>
>		By the end of 2007 a new free and open source framework should be available to help testing TAPIR provider implementations. This work is being carried out by RenatoDeGiovanni. The framework will:

  • Be compliant with the latest TAPIR XML Schema and specification.
  • Include an extensible set of test packages that can be run against TAPIR end points. This will enable new test packages to be added at any time. Test packages will be generic or related to specific data provider profiles, such as assuming that the provider is using a particular database or supports specific query templates.
  • Include at least one generic test package that should go through all available operations performing essential tests as well as XML Schema validation.
  • Include at least one test package to test the search and inventory operations on providers that are using a specific database and that have mapped the new DarwinCore? conceptual schema with its two extensions (geospatial and curatorial).

A simple Web interface will also be developed to interact with the framework. Users will be able to input a TAPIR end point and select the desired test packages to be performed. Results will be displayed as clear messages associated with the result of each individual test.

 

The Big Dig

Revision 105 Nov 2007 - Main.RogerHyam

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="WebHome"

QualityAssurance

The TAG has a role in ensuring the quality of TDWG products. The management of quality is a well established discipline in the software engineering community. This page is a focus for efforts at introducing best practice into our processes.

Quality Assurance for TDWG Standards and Software

JimGraham has produced a useful document describing the software lifecycle and how it may be applied to the TDWG process. It is highly recommended that those involved in development of TDWG standards and other products review this document.

Discussion is needed as to which elements from this lifecycle should be enforced in TDWG processes.

Enforcing Quality Assurance Processes

The executive committee is required to ask the TAG its opinion on the formation of new Task and Interest Groups and also on the passing of new standards. The TAG needs a framework for expressing this opinion. People submitting new group charters and standards need to know what the TAG will be looking for when it assesses their proposal

The following is a provisional list of criteria by which any proposal will be assessed (partially based on Jim's document above):

  • Audience - Is it clear who the stake holders and users are for this product?
  • Use cases & Scenarios - Is it clear how the the intended audience will use this product? What is it for?
  • Feasibility Studies - What evidence is there this is a feasible thing to do? Even if this is a feasibility study there should be some evidence suggesting there is a potential benefit.
  • Alternatives Analysis - What alternative approaches have been considered and why are they ruled out?
  • Test Plan - How will we know if the product meets its objectives? How will we know if it has stopped meeting its objectives?
  • Maintenance Plan - How will the product survive in the longer term? How will defects be handled? How will users report them?
  • Documentation - What documentation will be produced to support the product?
  • Training - What training is required?
  • Integration - How does this product integrate with other TDWG products and those of other groups?

Testing Related Projects

TAPIR Testing

The Big Dig

Linking Topics

META FILEATTACHMENT attachment="Quality_Assurance_1_2-1.pdf" attr="" comment="Quality Assurance for TDWG Standards and Software" date="1194262529" name="Quality_Assurance_1_2-1.pdf" path="Quality Assurance 1_2-1.pdf" size="62607" stream="Quality Assurance 1_2-1.pdf" user="Main.RogerHyam" version="1"
View topic | History: r5 < r4 < r3 < r2 | More topic actions...
 
This site is powered by the TWiki collaboration platform

Valid XHTML 1.0 TransitionalCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback