Index: doc/LaTeX/D-Soil Model - Test report/D-Soil Model - Test Report.tex =================================================================== diff -u -r830 -r874 --- doc/LaTeX/D-Soil Model - Test report/D-Soil Model - Test Report.tex (.../D-Soil Model - Test Report.tex) (revision 830) +++ doc/LaTeX/D-Soil Model - Test report/D-Soil Model - Test Report.tex (.../D-Soil Model - Test Report.tex) (revision 874) @@ -56,16 +56,16 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \newcommand{\VersionNumberPlusSpace}{\VersionNumber\xspace} -\newcommand{\VersionNumber}{16.2.1} +\newcommand{\VersionNumber}{17.1.1} -\newcommand{\VersionDSLCore}{16.1.5} -\newcommand{\VersionDSLFormsGeo}{16.1.5} -\newcommand{\VersionDSLFormsMap}{16.1.5} -\newcommand{\VersionDSLFormsStandard}{16.1.5} -\newcommand{\VersionDSLGeo}{16.1.5} -\newcommand{\VersionDSLGeographic}{16.1.5} -\newcommand{\VersionDSLGeoIO}{16.1.5} -\newcommand{\VersionDSLProba}{16.1.5} +\newcommand{\VersionDSLCore}{17.1.1} +\newcommand{\VersionDSLFormsGeo}{17.1.1} +\newcommand{\VersionDSLFormsMap}{17.1.1} +\newcommand{\VersionDSLFormsStandard}{17.1.1} +\newcommand{\VersionDSLGeo}{17.1.1} +\newcommand{\VersionDSLGeographic}{17.1.1} +\newcommand{\VersionDSLGeoIO}{17.1.1} +\newcommand{\VersionDSLProba}{17.1.1} \newcommand{\VersionDGSGef}{16.1.1.3} \newcommand{\VersionGefTwee}{3.1.2.1002} \newcommand{\VersionDevExpress}{12.2.13.0} @@ -79,8 +79,9 @@ \title{\ProgramName} \subtitle{} \manualtype{Test Report for version \VersionNumber} -\version{2.0} -\date{December 2016} + +\version{3.0} +\date{January 2017} \author{dr. V. Trompille} @@ -110,7 +111,14 @@ \versionii{2.0} \dateii{Dec. 2016} +\authoriii{dr. V. Trompille} +\revieweriii{dr.ir. J.G. van \newline Putten} +\approvaliii{ir. J. Icke} +\versioniii{3.0} +\dateiii{Jan. 2017} + + \status{final} \deltarestitle @@ -132,6 +140,8 @@ The full documentation on the program comprises the following documents. +\renewcommand{\arraystretch}{1.3} + \begin{table}[H] %\caption{xxx} %\label{xxx} @@ -157,15 +167,32 @@ \caption{Non-functional requirements \ProgramNamePlusSpace applicable to testing} \label{tab:NFREQs} \begin{tabular}{|p{15mm}|p{\textwidth-15mm-24pt}|} \hline \rowcolor[gray]{.8}\textbf{Nr.} & \textbf{Description requirement} \\ \hline + +\textbf{NFR3} & Data definitions will follow existing and emerging standards such as IRIS as much as possible. (must have). \\ \hline + +\textbf{NFR6} & Ringtoets and its computational core must perform sufficient error checks on the validity and completeness of data during import or input, as well as during a computation. Ringtoets needs to show warnings and error messages in an error message dialog, and also write them to a log file. The message text needs to be clear in what went wrong and where it went wrong (context). The message finally needs to supply suggestions for improvement wherever possible (read \ProgramNamePlusSpace for Ringtoets. ) \\ \hline + +\textbf{NFR12} & The user-interface may not cause crashes during regular usage. \\ \hline + +\textbf{NFR15} & A documented test bench of unit tests and integration tests is required, to guarantee that numerical results are reliable and to prove that new versions do not differ unintentionally from previous versions. This has been formulated in Dutch as: ''De generieke code coverage wordt gesteld op 80\%, met uitzondering van de Delta Shell Light componenten, waarvoor een code coverage van 60\% zal worden gehanteerd''. \\ \hline + +\textbf{R13} & Software product wordt getest conform test afspraken (dus inhoudelijke test en acceptatietest). Er is voor elke productie stap een helder testplan, in de testen wordt aangetoond dat de functionaliteit is gerealiseerd. \\ \hline + +\textbf{U123} & De WTI Software moet tot een eenduidig en reproduceerbaar resultaat leiden. \\ \hline + +\textbf{U124} & De WTI Software moet robuust zijn voor kleine variaties in de invoer. Onder 'robuust' wordt verstaan: nooit een software crash. Dus: ofwel een melding dat invoer onjuist is, ofwel een melding dat een som niet convergeert, ofwel een antwoord retourneert. \\ \hline + +\textbf{U129} & Alle componenten binnen een bibliotheek moeten foutcodes retourneren met een gestandaardiseerde (nog nader vast te stellen) betekenis of gebruiken excepties om fouten door te geven. \\ \hline + +\textbf{U131} & Om de bibliotheken te kunnen testen levert elke bibliotheek testsoftware mee in de vorm van unit tests. \\ \hline + +\textbf{U133} & De WTI tools worden standaard alleen uitgeleverd met een Nederlandstalige UI en met Nederlandstalige meldingen en rapportages. \\ \hline + +\textbf{U145} & De naamgeving van objecten, parameters, functies moet over alle applicaties heen consistent zijn. Voor dat doel moet gebruik worden gemaakt van een (nog op te stellen) totale verklarende WTI lijst van begrippen en symbolen. \\ \hline + + %NFR 2 & The regular conventions, standards, tools and libraries for Deltares Systems software will be used. (must have). -%NFR & Data definitions will follow existing and emerging standards such as IRIS as much as possible. (must have). Where possible emerging standards will be followed. For geometry a format has been used since the start of the development of Ringtoets which remained unchanged. \\ \hline -%NFR 12 & The user-interface may not cause crashes during regular usage. (must have) \\ \hline %NFR 14 & Consistency between the input data and the output data must be guaranteed. (must have) \\ \hline -%NFR 15 & ''De generieke code coverage wordt gesteld op 80\%, met uitzondering van de Delta Shell Light componenten, waarvoor een code coverage van 60\% zal worden gehanteerd.'' \\ \hline -R13 & Software product wordt getest conform test afspraken (dus inhoudelijke test en acceptatietest). Er is voor elke productie stap een helder testplan, in de testen wordt aangetoond dat de functionaliteit is gerealiseerd. \\ \hline -U124 & De WTI Software moet robuust zijn voor kleine variaties in de invoer. Onder 'robuust' wordt verstaan: nooit een software crash. Dus: ofwel een melding dat invoer onjuist is, ofwel een melding dat een som niet convergeert, ofwel een antwoord retourneert. \\ \hline -U131 & Om de bibliotheken te kunnen testen levert elke bibliotheek testsoftware mee in de vorm van unit tests. \\ \hline -U133 & De WTI tools worden standaard alleen uitgeleverd met een Nederlandstalige UI en met Nederlandstalige meldingen en rapportages. \\ \hline \end{tabular} \end{table} @@ -290,34 +317,34 @@ \begin{tabular}{|p{\textwidth-40mm-36pt}|p{20mm}|p{20mm}|} \hline \rowcolor[gray]{.8}\textbf{Class name} & \textbf{Number} & \textbf{Failed} \\ \hline \multicolumn{3}{|p{120mm}|}{\textbf{\textit{Namespace: Deltares.DSoilModel.Data.Tests}}} \\ \hline -SosSoilLayer1DTest & 5 & 0 \\ -DSoilModelIOTest & 22 & 0 \\ CenterCrestLocationTest & 2 & 0 \\ -SosSoilProfilesImporterTests & 7 & 0 \\ DSoilModelDataSourceRedirestorTests & 19 & 0 \\ -DSoilModelProjectTest & 46 & 0 \\ \hline -DSoilModelTransformerTests & 7 & 0 \\ \hline +DSoilModelIOTest & 23 & 0 \\ +DSoilModelProjectTest & 46 & 0 \\ +DSoilModelTransformerTests & 7 & 0 \\ +SosSoilLayer1DTest & 6 & 0 \\ +SosSoilProfilesImporterTests & 7 & 0 \\ \hline \multicolumn{3}{|p{120mm}|}{\textbf{\textit{Namespace: Deltares.DSoilModel.Forms.Tests}}} \\ \hline -DSoilModelPluginTests & 13 & 0 \\ -DSoilModelGeometryEditorTests & 28 & 0 \\ -DSoilModelContextTest & 49 & 0 \\ -DSoilModelSegmentGeometryEditorTests & 15 & 0 \\ BoringControlTests & 3 & 0 \\ -SegmentControlTests & 3 & 0 \\ -FilmStripTest & 4 & 0 \\ BoringLookup1DControlTests & 3 & 0 \\ -SosSoilProfile1DControlTest & 4 & 0 \\ -SoilProfile1DLookup2DControlTest & 3 & 0 \\ BoringLookup2DControlTest & 3 & 0 \\ -SosSoilLayer1DControlTests & 4 & 0 \\ -CptLookup2DControlTests & 3 & 0 \\ -CptLookup1DControlTests & 3 & 0 \\ +CenterCrestLocationPropertyControlTest & 3 & 0 \\ CptControlTests & 3 & 0 \\ +CptLookup1DControlTests & 3 & 0 \\ +CptLookup2DControlTests & 3 & 0 \\ +DSoilModelContextTest & 79 & 0 \\ +DSoilModelGeometryEditorTests & 28 & 0 \\ +DSoilModelPluginTests & 15 & 0 \\ +DSoilModelSegmentGeometryEditorTests & 15 & 0 \\ +FilmStripTest & 4 & 0 \\ PreconsolidationStressControlTests & 3 & 0 \\ -SpecificMechanismPointLocationControlTests & 3 & 0 \\ -CenterCrestLocationPropertyControlTest & 3 & 0 \\ -SoilSegmentSplitLocationControlTests & 3 & 0 \\ \hline -\textbf{Total} & \textbf{261} & \textbf{0} \\ \hline +SegmentControlTests & 3 & 0 \\ +SoilProfile1DLookup2DControlTest & 3 & 0 \\ +SoilSegmentSplitLocationControlTests & 3 & 0 \\ +SosSoilLayer1DControlTests & 4 & 0 \\ +SosSoilProfile1DControlTest & 4 & 0 \\ +SpecificMechanismPointLocationControlTests & 3 & 0 \\ \hline +\textbf{Total} & \textbf{295} & \textbf{0} \\ \hline \end{tabular} \end{table} @@ -332,14 +359,14 @@ \renewcommand{\arraystretch}{1.3} \begin{tabular}{|p{\textwidth-83mm-36pt}|p{23mm}|p{28mm}|p{32mm}|} \hline \rowcolor[gray]{.8} Namespace & \textbf{Class} & \textbf{Method} & \textbf{Statement} \\ \hline -Deltares.DSoilModel.Data \newline (see \autoref{fig:TestCoverageData}) & 100\% (11/11) & 85.4\% (129/151) & 82.9\% (874/1054) \\ \hline -Deltares.DSoilModel.Forms \newline (see \autoref{fig:TestCoverageForms}) & 84.4\% (27/32) & 52.3\% (293/560) & 64.6\% (3190/4937) \\ \hline -\textbf{Total} & \textbf{88.3\% (38/43)} & \textbf{59.3\% (422/711)} & \textbf{67.8\% (4064/5991)}\\ \hline +Deltares.DSoilModel.Data \newline (see \autoref{fig:TestCoverageData}) & 100\% (11/11) & 87.4\% (132/151) & 87\% (924/1062) \\ \hline +Deltares.DSoilModel.Forms \newline (see \autoref{fig:TestCoverageForms}) & 84.4\% (27/32) & 53.5\% (300/561) & 65.7\% (3249/4943) \\ \hline +\textbf{Total} & \textbf{88.4\% (38/43)} & \textbf{60.7\% (432/712)} & \textbf{69.5\% (4173/6005)}\\ \hline \end{tabular} \end{table} -The code coverage of \ProgramNamePlusSpace (\textbf{67.8\%}) is more than the minimum required code coverage of 60\% prescribed in \cite{OverallTestPlanWTI}. +The code coverage of \ProgramNamePlusSpace (\textbf{69.5\%}) is more than the minimum required code coverage of 60\% prescribed in \cite{OverallTestPlanWTI}. @@ -486,7 +513,7 @@ & & 31.04 & \OK \\ \hline \textbf{REQ 7.1} & \textbf{Stochastic subsoil information} & 41.01d & \OK \\ % Table Profielen - Kans -& & 41.01e & \OK \\ % Table Profielen - Profiel +& & 41.01e & \OK \\ \hline % Table Profielen - Profiel \textbf{REQ 8.2} & \textbf{Table view for soil parameters} & 12.01 & \OK \\ % Icons in table & & 12.03 & \OKbut \\ % Names of soil parameters @@ -529,7 +556,7 @@ & & 34.04a & \OK \\ % Delete reference line & & 34.04g & \OK \\ % Add reference line via right-mouse option & & 35.05a & \OK \\ % Synchronisation -& & 35.05b & \OK \\ % Synchronisation +& & 35.05b & \OK \\ \hline % Synchronisation \textbf{REQ 9.7} & \textbf{User can draw 2D soil layers} & 35.03 & \OK \\ \hline % geometry editor @@ -673,15 +700,36 @@ \begin{table}[H] \caption{Overview of the 'non-functional' requirements and their corresponding test} \label{tab:OverviewTestsNF} -\begin{tabular}{|p{10mm}|p{\textwidth-90mm-48pt}|p{70mm}|p{10mm}|} \hline -\rowcolor[gray]{.8} \textbf{REQ} & \textbf{Description} & \textbf{Test} & \textbf{OK?} \\ \hline -\textbf{R13} & \textbf{Perform test according to engagement} & A test plan explaining how the program should be tested is available \citep{DSoilModel_TestPlan} and this document (Test Report) shows that all the functionalities of the program are implemented and tested. & \quad \quad \OK \\ \hline -\textbf{U124} & \textbf{No crash} & Test-script nr. 47 of the Test Document (see appendix \autoref{app:TestDocument}) attests that no crash occured when performing all test-scripts listed in the Test Document (which represents about 8 hours working with \ProgramName), proving that this requirement is met. & \quad \quad \OK \\ \hline -\textbf{U131} & \textbf{Unit tests} & See \autoref{sec:2.1} for an overview of the unit tests. & \quad \quad \OK \\ \hline -\textbf{U133} & \textbf{UI (incl. messages) in Dutch language} & All the test-scripts of the Test Document (see appendix \autoref{app:TestDocument}) checks that the name, hint and unit of each UI field and also all validation and log messages are in Dutch. Only few missing translations have been found in some testscripts. & \quad \quad \OK \\ \hline +\begin{tabular}{|p{10mm}|p{\textwidth-90mm-48pt}|p{10mm}|p{70mm}|} \hline +\rowcolor[gray]{.8} \textbf{Nr.} & \textbf{Requirement} & \textbf{Met} & \textbf{Description/remark} \\ \hline + +\textbf{NFR3} & Data definition & Yes & See U145. \\ \hline + +\textbf{NFR6} & Error checks on the validity and completeness of data & Yes & See REQ 9.14 in \autoref{tab:OverviewTests}. \\ \hline + +\textbf{NFR12} & No crashes during regular usage & Yes & Test-script nr. 47 of the Test Document (see appendix \autoref{app:TestDocument}) attests that no crash occured when performing all test-scripts listed in the Test Document (which represents about 8 hours working with \ProgramName), proving that this requirement is met. \\ \hline + +\textbf{NFR15} & A documented test bench of unit tests and integration tests is required & Yes & Test coverage of \ProgramNamePlusSpace is 69.5\% (see \autoref{sec:CodeCoverage}). \\ \hline + +\textbf{R13} & Perform test according to engagement & Yes & A test plan explaining how the program should be tested is available \citep{DSoilModel_TestPlan} and this document (Test Report) shows that all the functionalities of the program are implemented and tested. \\ \hline + +\textbf{U123} & Reproducible results & Yes & Tested in test script 46. \\ \hline + +\textbf{U124} & Robustness & Yes & See NFR12. \\ \hline + +\textbf{U129} & Return error messages & Yes & See REQ 9.14 in \autoref{tab:OverviewTests}. \\ \hline + +\textbf{U131} & Unit tests & Yes & See \autoref{chapter2} for an overview of the unit tests. \\ \hline + +\textbf{U133} & Dutch language & Yes & All the test-scripts of the Test Document (see appendix \autoref{app:TestDocument}) checks that the name, hint and unit of each UI field and also all validation and log messages are in Dutch. Only few missing translations have been found in some test-scripts.\\ \hline + +\textbf{U145} & Names and symbols are conform to WTI list. & Yes & The names of the input parameters used in test-scripts 12.03 are the names given in the WTI list. In these test-scripts, the correctness of the parameter name is checked. As this test-script pass, it can be concluded that the names are conform the WTI parameter list \citep{WTIParameterList}. No symbols are used in \ProgramName. \\ \hline + \end{tabular} \end{table} + + \section{Conclusion} All the '\textit{must-have}' requirements have been implemented. @@ -765,7 +813,7 @@ \renewcommand{\arraystretch}{1.2} -All the unit and integration tests performed for the validation of \ProgramNamePlusSpace pass and cover 67.8\% of the code, so more than the minimum required code coverage of 60\% prescribed in \cite{OverallTestPlanWTI}. +All the unit and integration tests performed for the validation of \ProgramNamePlusSpace pass and cover 69.5\% of the code, so more than the minimum required code coverage of 60\% prescribed in \cite{OverallTestPlanWTI}. A complete test of the User Interface has been performed by means of test scripts (see appendix \autoref{app:TestDocument}). Almost all of the test scripts pass; however, a few minor inconveniences were found and reported in the \textit{Known Issues} list. @@ -801,22 +849,22 @@ \begin{figure}[H] \centering - \includegraphics[width=1.4\textheight]{pictures/TestCoverageAll.png} + \includegraphics[width=220mm]{pictures/TestCoverageAll.png} \caption{Overall code coverage of \ProgramName} \label{fig:TestCoverageAll} \end{figure} \begin{figure}[H] \centering - \includegraphics[width=1.4\textwidth]{pictures/TestCoverageData.png} + \includegraphics[width=220mm]{pictures/TestCoverageData.png} \caption{Overview of the code coverage of Deltares.DSoilModel.Data} \label{fig:TestCoverageData} \end{figure} \begin{figure}[H] \centering - \includegraphics[width=1.4\textwidth]{pictures/TestCoverageFormsAll.png} - \includegraphics[width=1.4\textwidth]{pictures/TestCoverageForms.png} + \includegraphics[width=210mm]{pictures/TestCoverageFormsAll.png} + \includegraphics[width=210mm]{pictures/TestCoverageForms.png} \caption{Overview of the code coverage of Deltares.DSoilModel.Forms} \label{fig:TestCoverageForms} \end{figure}