Certification Plan

[rbonham]

Hi Everyone,

So I have some question I want to put out there for the group. I am not sure who if anyone will have the answers to these but I think they are things we should be considering..

1. Who is going to run the certification test for each MLS?

2. What are the steps to take if we connect to a MLS's certified RETS server and find what we believe to be a certification problem? More specifically we address this with the MLS/vendor and they claim it is not a problem, do we have an independent review/arbitration type of process?

3. Seem this is now a MLS policy, I assume someone at NAR is going to need to know which MLS are or are not certified how is this going to be communicated and kept updated.

4. What happens when we certify a MLS RETS server and 1 week later they roll out a software patch that breaks the certification. Is certification test tied to the product version of the RETS server? If so how is this monitored?

5. One of the original goals of certification was making the life of Realtor that wants to use this data easier. How are our certification tests going to do that? Specifically we talked about running test as a superuser.. This will not help agents as they will most certainly not be superusers.. How do we test different roles?

6. Can we and should we define usability test? For example a RETS IDX certification test.. This test would look for specific items needed to successfully use RETS to get IDX data. GetObject Support, HasKeyIndex, etc.. Also testing for things like is the server filtering IDX optout listing out of the feed, are excluded addresses filtered out.. Things that ideally should happen at the server and not be done by the agent who may just ignore the rules and place everything online. I see this type of test being an option certification test that we encourage MLS to pass...

I know some of these items require input from NAR, and the MLS policy committee however they are just some of my concerns and things I think we probably need to be looking at far more before the December meetings.

[LukeJD]

I think a lot of these problems could be solved with an automated system. Perhaps a website could be constructed where MLS's can submit their RETS feed for certification by providing a logon credential. An application will go through all submitted applicants testing their server in key areas for certification. If it passes they go in a database where they are automatically checked on a regular basis and the compliant, now certified, MLS's can be posted online. You could build a simple appeal process if the MLS is declined, where the MLS in question could be reviewed manually. I imagine a system like this wouldn't be hard at all to build and would reduce the work load considerably.

[x-sol]

If there was an automated system that policed the serves to ensure they remain certified after passing I think there would need to be options to notify the mls, the vendor, or simply send n-notifications.

From a vendor perspective I wonder how often it would be mls configuration error that bubbles up to the vendor and is investigated by the vendor.

#5 On our server(s) MLS staff can configure access of every data set on a per user basis or share the settings between users at every Resource:Class combination. Including available fields, result limitations, etc You would have to test each login which would require some type of special transaction in RETS I imagine.

#6 Yes. But, I think data validity is probably more critical. Does a number only contain 0-9 . - full data type validation. I saw a Boolean that had MaxLength of 1 say "False" and "True" the other day.... That is a pretty intense test requiring: data sampling, inspection, metadata cross reference etc.

I think if there was a test for IDX that was graded from A-F I would recommend mls staff to run it after creating a user; there should be a way to score and scope the results to the user and not the server/mls. I.e. track results for super user (server) and the results for other users; could even be used to create over all score stats. And, if the MLS staff setup a set of feeds and it scored A++ in my system at least they could use this A++ set over and over. Possibly even going on to build "roles" that were all A++ provided they could test them as such without fear of invalidating the entire MLS.