Chat History with WTANZ (#olivernz/$4953690b02c23702)

Created on 2010-04-18 20:56:57.

2010-04-18

Oliver Erlewein: 18:31:31

Ok.... I think let's start

Keis: 18:31:45

@Oliver, hello :)

Oliver Erlewein: 18:32:13

I'll first post the Mission

Oliver Erlewein: 18:32:37

WTANZ session 01
SESSION DETAILS:
18. April 2010
NZ time: 6:30PM
AUS east coast: 4:30PM
Website Tested:
http://www.sorted.org.nz/calculators/mortgage-repayments/page1.php
Mission:
1) Have a look at the mortgage calculator and roughly script/try some
tests to verify functional correctness. Try and be creative about
breaking the solution. (rough time estimate 40min)
2) Have a look and think at usability and if this calculator matches
your expectations or not. (rough time estimate 10min)
3) Have a look at browser compatibility and make a quick assessment.
(rough time estimate 10min)
Note:
Please do not...
* ...automate your testing
* ...performance test
* ...security test
* ...test the Login functionality. Just the mortgage calculator.
Aim of this exercise is to show & share different methodologies on how
to test something. There is no right or wrong here. Do what you
usually do and share your process in the discussion bit at the end.
Don't be shy to share. Ask for help if you don't understand something
or want more details!

Oliver Erlewein: 18:32:51

Dang, sent the whole thing withour CRs.

Marlena Compton (WTANZ,AUS): 18:33:01

I see a pirate in our midst...

Parimala Shankaraiah (WT, India): 18:33:03

Hi James and Richard <ss type="smile">:)</ss>

Marlena Compton (WTANZ,AUS): 18:33:06

could it be????

Oliver Erlewein: 18:33:09

You can also see it online here: http://groups.google.co.nz/group/software-testers-new-zealand/ browse_thread/thread/6cbee3a6fb040310

Richard Robinson (WTANZ): 18:33:17

ooo, look who it is, Mr Bach I presume

Oliver Erlewein: 18:33:18

Hi james

Oliver Erlewein: 18:33:31

I thought you'd go for brekki first?!

James Bach: 18:33:34

I actually just woke up

James Bach: 18:33:41

but then I got curious

James Bach: 18:33:46

no coffee

James Bach: 18:33:58

I'm an ash refuge today

Oliver Erlewein: 18:34:04

Ok. so everyone have a read

Oliver Erlewein: 18:34:37

Missions 2 & 3 are optional today. We'll see how we go on 1)

James Bach: 18:35:16

Hello Marlena

Marlena Compton (WTANZ,AUS): 18:36:07

Hi James :)

Oliver Erlewein: 18:36:08

The website is pretty easy to understand and I don't expect to find any major bugs. BUt there are some curiosities. The aim of the exercise is to see different styles of testing or how you'd go about setting out to test this.

Oliver Erlewein: 18:37:00

So first off please start by asking questions about the Mission. Is there anything unclear or is there a problem with the instructions?

Marlena Compton (WTANZ,AUS): 18:38:37

So are we primarily testing for basic functionality?

Oliver Erlewein: 18:39:16

Yes. As always there are no specs but I think the subject is probably uderstood.

James Bach: 18:39:47

results don't match with this: http://www.bbc.co.uk/homes/property/mortgagecalculator.shtml

James Bach: 18:40:24

100,000, 25 years, 6% comes out with slightly different answers on either one

Oliver Erlewein: 18:40:32

Aha!

Oliver Erlewein: 18:40:37

So James, why is that?

Oliver Erlewein: 18:40:52

Is that a bug or not?

James Bach: 18:41:38

It's a small difference. Might not matter, since this is just an estimator

James Bach: 18:42:08

but it makes you wonder whether one is doing continuous compounding and the other is not

James Bach: 18:42:35

Also, it suggests that we might compare with several other online calculators

James Bach: 18:42:39

and maybe Excel

Oliver Erlewein: 18:42:49

Yes indeed, good idea.

James Bach: 18:43:09

of course the currency is different, but that shouldn't matter, right?

Oliver Erlewein: 18:43:58

There might be another reason....

Richard Robinson (WTANZ): 18:44:45

I compared with my own mortgage. This site wants me to make higher repayments??? only by $2, but still a difference. Could be a compliance matter with their internal calculations....hmmm

Oliver Erlewein: 18:45:02

If you think something is a bug then you can raise it on http://bugrepository.com in the WTANZ01 project.

Oliver Erlewein: 18:47:30

Have you seen that there is the nominal & today's dollars thing on page 2?

sharath (WT,India): 18:47:53

My internet connection speed as per <a href="http://us.mcafee.com/root/speedometer/test_0150.asp">http://us. mcafee.com/root/speedometer/test_0150.asp</a> is 60KBPS. It&apos;s over 4 min now and still the page <a href="http://www.sorted.org.nz/calculators/mortgage-repayments/page1.php ">http://www.sorted.org.nz/calculators/mortgage-repayments/page1.php& lt;/a> is not displayed on firefox3.6 or ie 8.0

James Bach: 18:48:04

interesting that the two calculators match perfectly for $1,000,000, 1000%, and 40 years.

Parimala Shankaraiah (WT, India): 18:48:11

<quote author="olivernz" conversation="#olivernz/$4953690b02c23702" timestamp="1271573313" authorname="Oliver Erlewein" guid="xaaec173462a87de0d3de7dd0c34c3ff6473e46dcbfc7c5ff34609daaf00dce80" ><legacyquote>[12:18:33 PM] Oliver Erlewein: </legacyquote>If you think something is a bug then you can raise it on http://bugrepository.com in the WTANZ01 project.<legacyquote>

&lt;&lt;&lt; </legacyquote></quote>

James Bach: 18:48:13

yeah, I saw that, Oliver

James Bach: 18:48:26

doesn't match that either

Oliver Erlewein: 18:48:32

@sharath: I'd see that as a bug but a minor one as this is built for NZ users.

Oliver Erlewein: 18:49:01

@James: Getting there ;-)

Oliver Erlewein: 18:49:25

Tip: check validation of input on this app.

Richard Robinson (WTANZ): 18:49:29

if you put in a loan amount of 0.005 you dont have to pay any interest...

Richard Robinson (WTANZ): 18:49:30

0.005, 9%, 10yrs

Marlena Compton (WTANZ,AUS): 18:49:49

I don't think there's much validation of input going on here at all.

Oliver Erlewein: 18:50:49

@Richard: So what do you think is the bug? What would be the right reaction? Is this usability or functional?

Richard Robinson (WTANZ): 18:51:21

i think the developers and business would look at me strangely if i raised this as a bug

Marlena Compton (WTANZ,AUS): 18:52:25

no big deal :o)

Marlena Compton (WTANZ,AUS): 18:52:32

oops, meant :)

Oliver Erlewein: 18:52:33

the mission again:

Oliver Erlewein: 18:52:34

http://groups.google.co.nz/group/software- testers-new-zealand/browse_thread/thread/6cbee3a6fb040310

James Bach: 18:53:30

the difference between the two calculators I'm using increases as you lower the interest rate

Oliver Erlewein: 18:54:06

@Richard: Really?! I'd say it's a big usability bug. Try big values

James Bach: 18:55:26

okay, I really need some coffee. But I suggest that someone compare results with at least two other mortgage calculators, focusing on low interest rates and perhaps graphing several data points

Richard Robinson (WTANZ): 18:55:34

functionally i mean. im not onto usability. still too interested in playing with this thing

Parimala Shankaraiah (WT, India): 18:55:49

<quote author="satisfice" conversation="#olivernz/$4953690b02c23702" timestamp="1271573936" authorname="James Bach" guid="xd41c9af87e739bf5a34878cba9624bce753dc7bca653bb58f35136905f9f4c58" ><legacyquote>[12:28:56 PM] James Bach: </legacyquote>okay, I really need some coffee. But I suggest that someone compare results with at least two other mortgage calculators, focusing on low interest rates and perhaps graphing several data points<legacyquote>

&lt;&lt;&lt; </legacyquote></quote>

Marlena Compton (WTANZ,AUS): 18:55:56

thanks for joining us, despite the lack of caffeine.

Oliver Erlewein: 18:56:02

Thanks James! Have a good one and drop me a message when the cloud clears. My parents are about to travel too.

Oliver Erlewein: 18:57:15

Ok. so we have noticed that the calculators are different. I'd say lets raise a bug for that just noting that they are different for now.

Oliver Erlewein: 18:57:49

Let's also cover some of the other things like boundary and failure conditions

Oliver Erlewein: 18:57:50

?

Oliver Erlewein: 18:58:38

Who has tested it against other calculators that were different?

Marlena Compton (WTANZ,AUS): 18:59:38

I chucked in 2035 for the start date, 2010 for the end date and it gave me -$77,000 in interest.  Who needs the lottery.

Oliver Erlewein: 18:59:57

@Marlena: Good.

Oliver Erlewein: 19:00:11

@Marlena: Is that a defect?

Marlena Compton (WTANZ,AUS): 19:00:37

Yes, I would say so.  I'm adding it to the bug repository too.

fvaswani: 19:00:44

I'd say user shou;dn't be allowed to do that

Oliver Erlewein: 19:00:49

@Marlena: Excellent. Thanks!

fvaswani: 19:01:06

there should be some message to the user

Marlena Compton (WTANZ,AUS): 19:01:31

@fvaswani yes, I agree with you.

Oliver Erlewein: 19:02:13

@fvaswani: I think so.

Oliver Erlewein: 19:02:36

Is there a failure situation in this calculator?

fvaswani: 19:02:53

Similar happens for 'Amount of the mortgage' = a negative value

Oliver Erlewein: 19:03:05

Is there a combination that causes any warning?

Marlena Compton (WTANZ,AUS): 19:03:27

I haven't been able to find one, and I've been looking.

Richard Robinson (WTANZ): 19:03:38

i cant find any warnings eithd

sharath (WT,India): 19:03:38

I have disconnected/connected to the internet; restarted the PC, but unable to access the webpage. I am able to open all other web pages like google, Citibank, <a href="http://www.tourism.net.nz/">http://www.tourism.net.nz/</a> ; but not <a href="http://www.sorted.org.nz/">http://www.sorted.org.nz/</a> <ss type="cry">;(</ss>

Oliver Erlewein: 19:04:08

@sarath: That is sad. Sorry about that but I can't influence that.

Parimala Shankaraiah (WT, India): 19:04:13

@Sharath - how about checking your firewall?

fvaswani: 19:04:26

Negative loan amount and negative interest rate causes positive interest!

Oliver Erlewein: 19:04:31

@sarath: Or some webfilter of your ISP is filtering the page.

sharath (WT,India): 19:04:38

@sharath i have disabled firewall for now

Oliver Erlewein: 19:04:48

@faswani: Good! Bug?

fvaswani: 19:04:56

Therefore there is just simple calculation behind the calculations, but not check on input values

sharath (WT,India): 19:05:00

sry for the typo, it was for PAri

fvaswani: 19:05:15

More of a bug to have chekc on input values

Ram: 19:06:20

just figuring out things here for joining late - noticed @sharath 's msg for link; 'm on that site; now which mortgage calculators are you all looking at?

Oliver Erlewein: 19:06:34

@faswani: Would you have one massive bug report or several for each check you'd expect? Can you raise a defect like you'd raise it in real life on bugrepository? We'll discuss it after the testing time ends

fvaswani: 19:06:51

@Ram where are you based?

Oliver Erlewein: 19:07:14

@Ram: Welcome. Instructions: http://groups.google.co.nz/group/software-testers-new-zealand/ browse_thread/thread/6cbee3a6fb040310

fvaswani: 19:07:19

@Sharath probably no access outside ANZ maybe?

Ram: 19:07:33

thank you @oliver

Parimala Shankaraiah (WT, India): 19:07:37

@Farid, I am in Bangalore and can access it on IE, FF and Chrome <ss type="smile">:)</ss>

Oliver Erlewein: 19:07:42

@Ram: Just one calculator on the sorted.org.nz page. Others have been comparing it to results from others.

Oliver Erlewein: 19:09:09

Has anyone raised a defect under WTANZ01 yet?

Marlena Compton (WTANZ,AUS): 19:09:20

I did, for the problem with years.

Oliver Erlewein: 19:09:30

hmmm.. can't see it....

sharath (WT,India): 19:09:33

This is interesting -  I can access <a href="http://www.sorted.org.nz">http://www.sorted.org.nz</a>, and then calculators page from there I can choose mortgage repayment

Marlena Compton (WTANZ,AUS): 19:09:33

If anyone needs help with that let me know.

Parimala Shankaraiah (WT, India): 19:09:58

<quote author="sharathbyregowda" conversation="#olivernz/$4953690b02c23702" timestamp="1271574781" authorname="sharath" guid="xdf490f962756a2e52969812f7632113456827a1a0c697fc5cc47d77b5addeed7" ><legacyquote>[12:43:01 PM] sharath: </legacyquote>This is interesting -  I can access http://www.sorted.org.nz, and then calculators page from there I can choose mortgage repayment<legacyquote>

&lt;&lt;&lt; </legacyquote></quote>

Oliver Erlewein: 19:10:01

Yes, I can see it now.

Oliver Erlewein: 19:10:42

Ok. 40 mins are gone. Do we want to continue on with functional testing or move on to usability?

Oliver Erlewein: 19:11:31

Usability would encompass such things as Accessability, ease of use, fit for purpose,....

Richard Robinson (WTANZ): 19:12:07

good idea

Ram: 19:12:33

negative 1mil percent interest tells me my monthly repayment is -166,666,666.67 for $200K

Oliver Erlewein: 19:13:19

For fuctional testing: what about non numeric values? Anyone tried that?

Marlena Compton (WTANZ,AUS): 19:13:35

I did.

Oliver Erlewein: 19:13:41

And? no errors?

Oliver Erlewein: 19:13:47

Or oddities?

Oliver Erlewein: 19:14:11

Anyone tried browser back, reload & the such?

Keis: 19:14:15

for non-numeric in the amount loan e.g., "a" it got treated as 0

Marlena Compton (WTANZ,AUS): 19:14:25

I typed a  semi-colon into 1 of the years fields and it accepted it.  I would owe $20,100,000.00 in interest.

Keis: 19:14:29

for date fields, non-existent ones were allowed

Oliver Erlewein: 19:14:31

@Keis: What about 1000a000?

Dhara (WT, India): 19:14:32

yes, for non numeric, no validationsa

Keis: 19:15:42

@oliver, i tried that... it became 1000.00... kinda similar to when i tried 1.1.1 which got treated as 1.10

Keis: 19:15:57

maybe the non-numeric values at the end got disregarded

Dhara (WT, India): 19:16:04

also ,it appears, not max limits set for amounts digits

Oliver Erlewein: 19:16:27

@Keis/Dhara: So are those defects?

Ram: 19:16:43

looks like Zero is considered a negative number on this calculator

Oliver Erlewein: 19:16:45

or a problem of specifications?

Oliver Erlewein: 19:16:56

@Ram: Wow, good find.

Keis: 19:17:00

weird... i entered "10000000000000000000000000" for interest rate and it became "10,000,000,000,000,000,905,969,664.00%" in the results

Oliver Erlewein: 19:17:16

@Ram: Raise it as a defect on bugrepository.org project WTANZ01

Oliver Erlewein: 19:17:40

@Keis: So what's the issue? Any ideas?

Oliver Erlewein: 19:17:59

Ok. Any takers on usability?

Dhara (WT, India): 19:18:03

specifications should have mention proper validation messages

Marlena Compton (WTANZ,AUS): 19:18:20

I would need some hints for usability.

Richard Robinson (WTANZ): 19:18:46

Some fields could be tied down a bit better. eg i can enter -100 for the year on the date

Ram: 19:18:54

is it http://www.bugrepository.org ?

Oliver Erlewein: 19:18:59

@Marlena: Say you look at this from the perspective of a visually challanged person.

Oliver Erlewein: 19:19:09

@Ram: .com

Ram: 19:19:15

ok

Oliver Erlewein: 19:19:36

@Marlena: Can you change font size? What if you change font size in browser?

Marlena Compton (WTANZ,AUS): 19:19:56

So in that case, would the "calculate" button not have enough contrast?

Oliver Erlewein: 19:20:13

@Marlena: Does it follow NZ EGIF guidelines? <--- which you'd know nothing about being in AUS

fvaswani (WT, India): 19:20:23

I'd say Usability and Accessibility are 2 different things.

Oliver Erlewein: 19:20:35

@Marlena: Good. Can you switch to high contrast view?

Marlena Compton (WTANZ,AUS): 19:20:41

I think Sharath did our accessibility testing.

sharath (WT,India): 19:20:47

<ss type="smile">:)</ss>

Oliver Erlewein: 19:22:02

@fvaswani: Yes you could argue that. Accessibility here in NZ is usualy a sub category of Usability.

Oliver Erlewein: 19:22:33

@fvaswani: But as JB always says, that's just me ;-)

Oliver Erlewein: 19:22:46

...and it might be wrong.

Oliver Erlewein: 19:23:00

Ok. little behind on time.

Marlena Compton (WTANZ,AUS): 19:23:01

@Oliver not sure how to set that on my Mac

fvaswani (WT, India): 19:23:03

@Oliver yep agreed, and i guess Akl (NZ) is the same :)

Richard Robinson (WTANZ): 19:23:26

$100, 10%, 10999 Years gives you $inf interest to pay overall in todays dollars

Oliver Erlewein: 19:23:27

Task 3) Browser Testing

Marlena Compton (WTANZ,AUS): 19:23:46

7 minutes left for testing

Oliver Erlewein: 19:24:18

@Richard: How did you enter 10999 years?

Richard Robinson (WTANZ): 19:24:29

actuallly it was 10099

Oliver Erlewein: 19:24:56

@Richard: The field is limited to 4 digits so how did you enter 5?

Richard Robinson (WTANZ): 19:25:05

the most years i can get are 10998

Parimala Shankaraiah (WT, India): 19:25:16

@Everyone - a small tip for Browser testing - You could use <a href="http://browsershots.org/">http://browsershots.org/</a> to test with different browsers!

Richard Robinson (WTANZ): 19:25:25

well once i did the boundary of zero as the year

Richard Robinson (WTANZ): 19:25:32

i did a negative... -999

Oliver Erlewein: 19:25:35

@Parimala: That was the big clue ;-)

Oliver Erlewein: 19:25:46

@Parimala: Thanks for mentioning it

fvaswani (WT, India): 19:25:51

Guys would it be correct to say that most of the functional issues above could be fixed/avoided by just having input check?

Parimala Shankaraiah (WT, India): 19:26:05

@Yeah - 10 mins is too short for testing browsers, but this website could help. Using the right tools at the right time <ss type="smile">:)</ss>

Oliver Erlewein: 19:26:13

@faswani: Most likely.

Richard Robinson (WTANZ): 19:26:17

@ Fvaswani: I agree

Richard Robinson (WTANZ): 19:26:31

It would also give us errors to check against

Richard Robinson (WTANZ): 19:26:37

but i guess the app carries little risk

Oliver Erlewein: 19:26:53

@faswani: Although we have one thing to watch out for and that is different results from different calculators.

Oliver Erlewein: 19:27:14

Has anyone tried browsershots?

Richard Robinson (WTANZ): 19:27:25

yes ive used it a number of times

sharath (WT,India): 19:27:44

nrowser shots is fantastic

Oliver Erlewein: 19:28:07

@Richard: Have you tried using FireFox plugin Tamper Data to change values over and above the limits of the web page restrictions?

Ram: 19:28:11

just looking over browsershots... looks great

Oliver Erlewein: 19:28:41

Browsershots is great and probably one of the quickest ways to get a feel for browser compatibility.

Richard Robinson (WTANZ): 19:28:53

Ah true. I was thinking of saving the site locally, and manipulating the values being sent. But I thought that might be a bit mean to the DB/Site.

Marlena Compton (WTANZ,AUS): 19:28:55

I'm trying browsershots, it's cool.

Keis: 19:29:05

now was a bad time to try out ipadpeek.com.. it distracted me

Oliver Erlewein: 19:29:06

It has drawbacks though like no testing with posted data

Richard Robinson (WTANZ): 19:29:16

@Keis: Lol

Oliver Erlewein: 19:29:44

@Kais: yes ;-) ....but I'll check it out a8r

Keis: 19:30:26

for usability, do the links 'nominal dollars' and 'today's dollars' do anything?

Richard Robinson (WTANZ): 19:30:28

i tried pasting a screen shot into the fields too... no luck

fvaswani (WT, India): 19:30:30

I'd say browsershots is good to get a quick and dirty check but  its good to have various environmments hosted locally to try out various tests/options?

fvaswani (WT, India): 19:30:34

What people say?

Parimala Shankaraiah (WT, India): 19:30:39

<quote author="olivernz" conversation="#olivernz/$4953690b02c23702" timestamp="1271575931" authorname="Oliver Erlewein" guid="x14e0f91be490b0aec653108be86d3820e1474aabc30a3a1def8872fa5de10038" ><legacyquote>[1:02:11 PM] Oliver Erlewein: </legacyquote>Browsershots is great and probably one of the quickest ways to get a feel for browser compatibility.<legacyquote>

&lt;&lt;&lt; </legacyquote></quote>

Oliver Erlewein: 19:30:43

Ok. officially this is the end of the testing phase. So I'd like you to get to an end and we'll stert discussing at xx:35.

Marlena Compton (WTANZ,AUS): 19:31:18

Be sure to log your bugs at bugrepository.com

Richard Robinson (WTANZ): 19:31:19

@Farid: yes, I agree. I only use Browsershots.org as one item in my checklist. It may highlight areas of weakness in the site, that I can further investigate. But I dont put too much effort into it. Thats just from my experience.

Oliver Erlewein: 19:32:02

Browsershots is Open Source so you could have your company internal version too. It's a bit tricky though.

Oliver Erlewein: 19:32:31

Another thing testers should always have is a number of vitual machines running different OS's and browsers.

Oliver Erlewein: 19:33:11

I suggest VirtualBox.org as a virtualisation platform but VMware, Parallels,.... are good too (just not free ;-))

Richard Robinson (WTANZ): 19:33:20

@Oliver: I love settign up different configs to sim user machines. It takes a lot of effort out of testing also. I recommend using orthogonal arrays to assist also

fvaswani (WT, India): 19:34:04

Yep Virtualbox is what we've gone with

Oliver Erlewein: 19:34:29

It also helps to have these VMs as a repository for all to use at the company.

Richard Robinson (WTANZ): 19:34:29

@Oliver: can I do performance testing from Virtualbox.org? I want to be able to check the load times of our corp website over time.

Parimala Shankaraiah (WT, India): 19:34:50

AllPairs tool at James Bach&apos;s website gives a fair amount of combinations to test - <a href="http://www.satisfice.com/tools.shtml">http://www.satisfice.com/ tools.shtml</a>

Oliver Erlewein: 19:34:52

@Richard: Let's take that offline.

Oliver Erlewein: 19:35:26

Ok all. Can I ask you to all stop testing? Let's now discuss what we've found.

sharath (WT,India): 19:35:34

chk this <a href="http://www.softwaretestingclub.com/forum/topics/how-can-i-test- something">http://www.softwaretestingclub.com/forum/topics/how-can-i- test-something</a>

Oliver Erlewein: 19:36:19

First I'd like you to feed back how you first addressed this session.

Oliver Erlewein: 19:36:38

i.e. what ideas did you have starting with a fresh and unknown product.

Oliver Erlewein: 19:36:53

did you have any plan on what you would do?

Oliver Erlewein: 19:37:01

did you just explore?

Oliver Erlewein: 19:37:08

who wants to be first?

fvaswani (WT, India): 19:37:14

not really. just explore

Richard Robinson (WTANZ): 19:37:23

Is it one at a time?

fvaswani (WT, India): 19:37:43

(one of the reasons was coz joined late)

Oliver Erlewein: 19:37:48

@richard: go for it if you like.

Ram: 19:37:52

@oliver - joining in late,i just started exploring this time; but i agree, planning initially on what the mission is and the approach to tackle the problems would be much  helpful as we go along.

Richard Robinson (WTANZ): 19:38:02

i think Ram is on his way

Oliver Erlewein: 19:38:16

Anyone who did some planning?

Richard Robinson (WTANZ): 19:38:23

Let me know when its my turn

Oliver Erlewein: 19:38:31

What did it look like (even if it proved not useful later on)?

Marlena Compton (WTANZ,AUS): 19:38:49

I decided to do everything in two browsers

Oliver Erlewein: 19:39:05

@Marlena: Interesting. Why?

Marlena Compton (WTANZ,AUS): 19:39:52

I've found that for some applications, there can be vast differences.  For others it is less so.  This application sucked equally across all browsers.

Oliver Erlewein: 19:39:55

Anybody have a testing checklist or other standard procedure they referenced to to solve this problem?

fvaswani (WT, India): 19:40:13

@Marlena like t hat idea. have seen people doing that a lot.

Oliver Erlewein: 19:40:17

@All: I think this is more a free for all.

Oliver Erlewein: 19:40:30

don't hold back

Richard Robinson (WTANZ): 19:41:09

My approach was to do ET to understand the App. I used data that was personal to me, to get a feel. AFter about 1 minute, the complexity of the app reduced, I was familiar, and I let rip with some field boundary tests.

Oliver Erlewein: 19:41:10

@Ram: what planning would you have done if you had time? any "tools"/methods you'd have used?

Richard Robinson (WTANZ): 19:41:35

Actually it was all ET. I kept a log in notepad with my tiny goals each step of the way

Oliver Erlewein: 19:41:36

@Richard: Sounds good. i like the use of personal data.

Keis: 19:41:40

initially, i wanted to figure out how it behaves with simple calculations. e.g., $100 loan amount for 10% interest (using simple math) ought to get $110 in total payment

Richard Robinson (WTANZ): 19:41:53

Peronal data = I am the oracle :)

Oliver Erlewein: 19:42:04

Personal data is data that is very familiar to you so errors would be easily spotted.

Oliver Erlewein: 19:42:42

@Keis: That is the second most likely approach I think. Use simple data and get a feel.

Oliver Erlewein: 19:42:55

@Keis: How did you verify the results?

Oliver Erlewein: 19:43:05

Did anyone here use Excel or calculator?

Keis: 19:43:26

@Oliver, i used an excel file and compared my computation against the total

Richard Robinson (WTANZ): 19:43:35

I used simple data across calculators. I compared to another banks calculator, and got a similar result to my personal data. Sorted seems to overestimate the repayment amounts slightly

Marlena Compton (WTANZ,AUS): 19:43:38

@Richard so tell us more about the tiny goals and ET.  I'm curious about this.

Keis: 19:43:47

but they mostly turned out false so i quit logging the stuff i try in excel

Keis: 19:43:58

false i.e., not equal

Oliver Erlewein: 19:44:39

I find it odd that a simple thing as calculating interest can have different results for different apps/Excel/calculator

Oliver Erlewein: 19:44:56

What do you's think can be reasons for that?

fvaswani (WT, India): 19:44:59

calculator at times. But i was looking more at boundry and conditional testing than actual calculations

Marlena Compton (WTANZ,AUS): 19:45:06

@Oliver that seemed like the biggest defect to me.

Oliver Erlewein: 19:45:12

i.e. What would you tell your developer to check for?

Oliver Erlewein: 19:45:42

@Marlena: Sorry, what's the biggest defect?

Marlena Compton (WTANZ,AUS): 19:46:17

@Oliver the difference in how interest was calculated, but then I guess that's also somewhat relative.

Ram: 19:46:28

@oliver, more than tools, certainly the planning approach would have been
a. understand the objective
b. Ask questions for ambiguities - in this case i noticed multiple mortage calculators, i wasn't sure which one the group was looking at.
c. plan on the approach for different types  of testing - including boundary, and edge scenarios
d. Scenarios - understand how mortgage concept works and come up with the approach / scenarios around the user context
e. plan on the tools and resources required to complete the mission
f. ESTIMATe what it takes for the planned approach - so that helps building the estimation skillls which we often need to develop for the task completion
g. attack the mission - well, there could still be situations where we might have to keep the plan aside and do whats needful - thats where the Risk planning and MItication would have helped;

Richard Robinson (WTANZ): 19:46:39

@ Marlena: Thanks. I thought of a basic checklist of ways the calculator be tested. I started with data entry into the fields using positive data, then random values. once i found there was no validation, i moved away from this area, there were no low yeilding fruit to pick. i looked at the source code for clues, and tried other banks calculators. I guess there were 3 items on my checklist: data entry boundaries & validation, source code and value processing, and comparison to the 'same' calculation in other establish sites

Keis: 19:46:56

i think there may be add'l factors i'm not aware of ... e.g., the total for an annual thing is not as straightforward as Amount*(1+interest)

Oliver Erlewein: 19:47:23

@Ram: Wow. Great share there. I like b). ;-)

Marlena Compton (WTANZ,AUS): 19:47:52

@Keis and now I'm wondering where they explain their calculation.

Oliver Erlewein: 19:48:07

@Richard: Good one to have a look at code. That can sometimes tell you a lot.

Richard Robinson (WTANZ): 19:48:25

@Ram: great ideas, i like them all

Oliver Erlewein: 19:48:37

@Marlena: Another great spot. i would have expected to have an explanation / formula somewhere explaining it.

Oliver Erlewein: 19:49:00

Did anyone run the page through an HTML checker like....

Oliver Erlewein: 19:49:16

http://validator.w3.org/ ?

Keis: 19:49:32

@oliver, @marlena yeah, i wanted to try to view source and find some hints on their calculation. but decided against it since it might take me longer to look for their code

Oliver Erlewein: 19:49:35

There's nothing major but some nigglies I'd push back to a developer.

Oliver Erlewein: 19:50:00

HTML validation is low haning fruit

Oliver Erlewein: 19:50:42

Another thing you might like to do is use FireFox with the HttpLiveHeaders plug-in running.

Marlena Compton (WTANZ,AUS): 19:50:43

@Oliver so does that validation still work if the app is using AJAX?

Oliver Erlewein: 19:50:54

@Marlena: Usually not

Oliver Erlewein: 19:51:08

LiveHTTPheaders that is! Sorry

Oliver Erlewein: 19:51:47

That plug in records all request and response headers of what you're doing. That gives you a first rudimentary log of what you did and the values you have POSTed

Richard Robinson (WTANZ): 19:52:05

I put myself in the shoes of a customer to the site, a developer of the site, and the site owner. I worked out where the money was coming from to pay for development, and the returns the site gets. This all helped me to weigh the risks of the conditions and helped me to attribute my time to each functional area

Oliver Erlewein: 19:52:58

Did anyone have luck in finding defects with browsershots.org?

Richard Robinson (WTANZ): 19:53:36

I did some boundary testing on a banks calculator, and the site kicked me out :( 'page not found'.... lol.

Oliver Erlewein: 19:54:01

@Richrd: Which site kicked you?

Richard Robinson (WTANZ): 19:54:08

starts with a B

Oliver Erlewein: 19:55:02

@Keis: Just seen your defect with the div zero. Great find!

Oliver Erlewein: 19:55:23

@All: See in bugrepository.com under WTANZ01

Keis: 19:55:43

@Oliver, thanks <ss type="blush">(blush)</ss>

Oliver Erlewein: 19:56:16

@dhara: Do you think the Safari 4.0 error is real or just browsershots.org not working?

Oliver Erlewein: 19:56:26

i.e. did you veriy that result?

Ram: 19:56:42

@oliver - i noticed several browser screenshots being empty for the given url

Oliver Erlewein: 19:56:50

I found that the site advertising doesn't work on FireFox 2.0

Dhara (WT, India): 19:56:59

i think it is given that result for that moment only

Ram: 19:57:02

i liked the idea of the website providing the screenshots of all the browsers it tries the URL with

Oliver Erlewein: 19:57:03

@Ram: That's brosershots problem.

Ram: 19:57:59

thought so, 'coz one of those browsers where it was empty was IE7, and i know it would not be an issue with displaying it on that browser. I use IE8 on my machine

Ram: 19:58:46

well, though i made an assumption in my prv msg, but it was a confident assumption on my part :)

fvaswani (WT, India): 19:58:50

Verdict? Browsershots not reliable?!

Oliver Erlewein: 19:58:59

Ok. can we have a quick round the "table" and feed back what you thought of the session and what was useful to you? Order please as seen on screen (i.e. Dhara 1st, then fvaswani....)

Oliver Erlewein: 19:59:26

@faswani: I'd rather say stay critical and don't always believe what you see.

Oliver Erlewein: 20:00:26

Oh, just got reprimanded that the order isn't the same for all.

Oliver Erlewein: 20:00:33

So I'll call them out.

Oliver Erlewein: 20:00:46

Dhara? Any comments?

Marlena Compton (WTANZ,AUS): 20:01:00

Not sure she's still with us

Oliver Erlewein: 20:01:03

Just say "pass" if you have nothing to add.

Dhara (WT, India): 20:01:36

First of all i must say i am gald to join the first ANZ WT session, I liked today's excercise (didnt have much time to compare with other cals, joined bit late

Dhara (WT, India): 20:01:55

i didnt know about browsershot site before

Parimala Shankaraiah (WT, India): 20:02:03

<quote author="dhara5" conversation="#olivernz/$4953690b02c23702" timestamp="1271577905" authorname="Dhara" guid="xf3d898277ad5ee25dbfa1655ba2d2e03a18dbd1fe99fbc2f8e10476347a9945b" ><legacyquote>[1:35:05 PM] Dhara: </legacyquote>First of all i must say i am gald to join the first ANZ WT session, I liked today's excercise (didnt have much time to compare with other cals, joined bit late<legacyquote>

&lt;&lt;&lt; </legacyquote></quote>

Dhara (WT, India): 20:02:06

Thanks Parimala for letting us know

Parimala Shankaraiah (WT, India): 20:02:24

@Dhara - you don't have to stay up late or wake up early to take part in BWT. WTANZ is live!

Dhara (WT, India): 20:02:59

I will join BWT too whenever possible :)

Dhara (WT, India): 20:03:21

and having ANZ WT is great too :)

Dhara (WT, India): 20:03:31

thanks all

Oliver Erlewein: 20:03:48

@Dhara: Thanks! I guess now it's fvaswani's turn.

Oliver Erlewein: 20:04:05

I am aware we're going overtime. Anyone here that has to leave?

Oliver Erlewein: 20:04:12

...and wants to go next?

Oliver Erlewein: 20:04:24

after fvaswani?

Richard Robinson (WTANZ): 20:04:25

I liked the open way of running this. Thanks Oliver. I appreciated the positive sentiment in the room, and the support given for each other. I liked the websites suggested for followup

Richard Robinson (WTANZ): 20:04:42

I think that discussing as we go is the way to go. I got a lot of value out of knowing what others were trying the same time as I was.

Marlena Compton (WTANZ,AUS): 20:04:43

we're going until 6:30 right?

Oliver Erlewein: 20:05:01

@Marlena: Sorry, my bad of course.

fvaswani (WT, India): 20:05:05

Cool... I got some good pointers from other testers esp the prep stuff from Richard and Ram

Keis: 20:05:06

I liked having counter- or additional ideas being raised. That's something you definitely can't have when you're testing on your own.

Oliver Erlewein: 20:05:07

8:30PM NZT

fvaswani (WT, India): 20:06:19

But i must confess my main intention to join was to find out what are the benefits of WT? How can it help one and all?

Oliver Erlewein: 20:06:26

@Richard: I was reluctant to give too much too early but I did like the "mashup" type style too.

Richard Robinson (WTANZ): 20:07:28

It was an organic approach

Oliver Erlewein: 20:07:35

@fvaswani: As an idea, instead of finding out why not help form what it can do for you? WT should be open to change and benefit the need and not follow a stoic protocol or something.

fvaswani (WT, India): 20:07:35

I understand as a tester it helps me see and learn from other testers and being a team lead/mngr it will also be helpful when i go hunting for people :)

Oliver Erlewein: 20:08:10

@fvaswani: Please no hunting! Testers are a rare species and we need each and everyone ;-)

Oliver Erlewein: 20:08:43

@Marlena: Any insights?

fvaswani (WT, India): 20:08:44

lol yep :) i meant change them from testers to a test analyst may be?

Ram: 20:08:54

@fvaswani @Oliver :)

Marlena Compton (WTANZ,AUS): 20:09:12

It's great to practice with other testers in this timezone!

Richard Robinson (WTANZ): 20:09:17

@Farid: thank you

Oliver Erlewein: 20:09:20

@Marlena: LOL

Keis: 20:09:27

@Marlena yeah! :)

Ram: 20:09:50

@Marlena agreed - helps learn different perspectives

Marlena Compton (WTANZ,AUS): 20:10:09

As far as the app...it was good to see people discussing how the mortgage calculation was comparing.  I always want to go straight off to boundary testing.

Richard Robinson (WTANZ): 20:10:12

@All: I was in the WT32 from Bangalore last night, it was terrific fun. And the team was very friendly.

Marlena Compton (WTANZ,AUS): 20:10:30

But made me think about that.

Richard Robinson (WTANZ): 20:10:43

I urge all to try the different groups.

Oliver Erlewein: 20:10:47

I had looked at this calculator for about 3hrs in prep for this and I still learned a lot from this.

Richard Robinson (WTANZ): 20:10:54

But always come back to this one :)

fvaswani (WT, India): 20:11:03

@Richar :)

Marlena Compton (WTANZ,AUS): 20:11:19

I've found each group has its own flavor.

Oliver Erlewein: 20:11:29

@Richard: I think the time zones take care of which groups you'll attend.

Oliver Erlewein: 20:11:45

@Parimala: Any feedback?

Marlena Compton (WTANZ,AUS): 20:12:10

Shows how diversity is helpful in technology.

Oliver Erlewein: 20:12:10

@sarath?

Oliver Erlewein: 20:12:34

I personally REALLY liked that we were all active in participating.

Parimala Shankaraiah (WT, India): 20:12:36

@Everyone, I was just watching over the session without testing. I do have some feedback in general about what I thought.

fvaswani (WT, India): 20:12:44

May be a Q's for @Parimala - what is the size of group you get normally? whats the min to keep it interesting and max to keep it managemable?

Oliver Erlewein: 20:13:22

@fvaswani: I think that depends much on the task at hand to.

Oliver Erlewein: 20:13:42

In this session I could have probably dealt with about another 5 testers

Oliver Erlewein: 20:13:56

more and it would have been crowded.

Oliver Erlewein: 20:14:09

But still lots of fun I would imagine ;-)

Parimala Shankaraiah (WT, India): 20:14:20

Firstly, it was great to see testers jump in right away with great testing oracles - like comparing with other mortgage applications. thinking about different OSes and browsers, even VMs. I also liked the fact that testers thoroughly asked great questions regarding the mission while they tested. I found the communication to be highly effective in this group. Best one in fact in terms of communication <ss type="smile">:)</ss> You did a great job guys. Well Done!

Parimala Shankaraiah (WT, India): 20:14:37

@We have atleast 3 testers in most sessions and have had upto 25 people in some sessions.

Parimala Shankaraiah (WT, India): 20:14:59

In sessions where we have more than 10 ppl, it will be facilitated by 2 facilitators. We plan it that way.

Marlena Compton (WTANZ,AUS): 20:15:47

I really like the communication in this group as well.

Marlena Compton (WTANZ,AUS): 20:16:21

It definitely helped me wrap my head around the exercise a little more.

Oliver Erlewein: 20:16:53

(sun) happy to have had such glowing feedback. thanks

fvaswani (WT, India): 20:17:26

<ss type="pizza">(pi)</ss> and herez the virtual pizza for all ;)

Ram: 20:17:34

yay! #pizza

Marlena Compton (WTANZ,AUS): 20:17:37

where's the beer?

Oliver Erlewein: 20:17:37

THANKS!!!!

fvaswani (WT, India): 20:17:46

<ss type="beer">(beer)</ss>

Oliver Erlewein: 20:17:47

(beer)

Parimala Shankaraiah (WT, India): 20:18:04

<ss type="party">(party)</ss>

Richard Robinson (WTANZ): 20:18:07

Well done everyone. I cant wait for the next one. That is a very good sign for me :)

Oliver Erlewein: 20:18:09

(d) <-- i was having wine

fvaswani (WT, India): 20:18:26

Cheers guys... it was great to catch up and meeting/talking to you all

Oliver Erlewein: 20:18:34

Ok. befor we wrap up

Marlena Compton (WTANZ,AUS): 20:18:37

So *that's* why you were so communicative.

Oliver Erlewein: 20:18:43

the next session will be in a fortnight

Parimala Shankaraiah (WT, India): 20:18:53

<quote author="olivernz" conversation="#olivernz/$4953690b02c23702" timestamp="1271578933" authorname="Oliver Erlewein" guid="x662bf6214edb9f7883ce5f5dd04d073d2445047a201539a36aa5fb4b22d837b9" ><legacyquote>[1:52:13 PM] Oliver Erlewein: </legacyquote>the next session will be in a fortnight<legacyquote>

&lt;&lt;&lt; </legacyquote></quote>

Keis: 20:19:09

Thanks to all <ss type="smile">:)</ss> I had an awesome time <ss type="smile">:)</ss>

Ram: 20:19:10

@oliver, ok - so thats may 1st

Ram: 20:19:11

sorry may 2

Richard Robinson (WTANZ): 20:19:14

i love the pencils

Oliver Erlewein: 20:19:15

that would then be the 2nd may

Marlena Compton (WTANZ,AUS): 20:19:17

fortnight = 2 weeks

Oliver Erlewein: 20:19:40

Probably same time same place but check the usual spots for announcements.

Parimala Shankaraiah (WT, India): 20:19:54

Thank you everyone

Oliver Erlewein: 20:20:00

I will also get a section for WTANZ on weekendtesting.com

Ram: 20:20:04

http://ramsblog.wordpress.com/2010/04/18/ wtanz-weekend-testing-australia-newzealand/
i haven't mentioned all the names, and more observation there, i will probably update the post when i get back to it

Oliver Erlewein: 20:20:11

but am not sure how fast that will be.

Oliver Erlewein: 20:20:42

@Ram: Wow!

Oliver Erlewein: 20:21:18

@Ram: Can I use that for posting on Weekendtesting.com?

Richard Robinson (WTANZ): 20:21:39

@Ram: amazing. Can I use you for testing on my projects? They will be done in half a day ;)

Ram: 20:21:42

sure, but i don't think it has all the details

Ram: 20:21:59

i missed some initially and a some during the session while i was looking at the calculator

Oliver Erlewein: 20:22:20

I will post the full session somewere soon ;-)

Oliver Erlewein: 20:22:32

@Ram: you got the main points though.

Ram: 20:22:43

@Richard - hehe sure- if time allows, why not :) great to be associating with you all

Marlena Compton (WTANZ,AUS): 20:22:44

@Ram  That's a really great post.  Very fast too :)

Parimala Shankaraiah (WT, India): 20:23:05

@Ram - That was lightning fast post!

Richard Robinson (WTANZ): 20:23:38

@Ram: what automation tool did u use for that post? :p

Ram: 20:23:43

@parimala thanks  - i use windows live writer to make notes during the session, so that helps to compile and publish

Ram: 20:24:02

@richard windows live writer :)

Richard Robinson (WTANZ): 20:24:13

@Ram: i take back my tongue... lol

Parimala Shankaraiah (WT, India): 20:24:20

@Ram - cool. I'll explore that shortly

Richard Robinson (WTANZ): 20:24:48

@Ram: well done, have a beer on me <ss type="hug">(hug)</ss>

Ram: 20:25:00

oh thank you @Richard

Parimala Shankaraiah (WT, India): 20:25:23

Bye Everyone. See you again in a fortnight <ss type="wink">;)</ss>

fvaswani (WT, India): 20:25:33

Ok see you soon guys. Thanks all.

Richard Robinson (WTANZ): 20:25:35

Thanks everyone, see you next time :)

Ram: 20:25:47

bye parimala, hopefully i will join next weeks bangalore WT

Keis: 20:25:49

Nice meeting you all :)

Oliver Erlewein: 20:26:00

Bye Parimala

Parimala Shankaraiah (WT, India): 20:26:09

@Ram. sure. Jon Bach&apos;s 2nd session is scheduled for next week <ss type="smile">:)</ss>

Marlena Compton (WTANZ,AUS): 20:26:14

CU Parimala

Parimala Shankaraiah (WT, India): 20:26:36

Bye. Happy Weekend. Well, its almost over <ss type="smile">:)</ss>

Oliver Erlewein: 20:26:37

Ok, I think this thread is now officially closed

Oliver Erlewein: 20:26:50

that means anything discussed here on won't be in the log.