HTTPS all the Things (Automated Edit)

classic Classic list List threaded Threaded
44 messages Options
123
Reply | Threaded
Open this post in threaded view
|

HTTPS all the Things (Automated Edit)

Bryce Jasmer
I have written a script that will search for OSM objects that have a website tag that explicitly states "http://..." or implicitly uses http by leaving of the protocol specification. The script will then loop through all that it discovers and asks the http site if it will redirect me to the secure version of the website over the https protocol. If it does, I will update the database with the new value.

This has a couple of advantages. From now through the end of time, any user clicking on one of those links will be spared the time it takes to establish the connection, ask if there is a secure version of the site, and tear down the connection. It's on the order of 10-200 ms to do, but over the life of the link and the number of objects that are clicked and the population, this could save centuries of time :-)

Another advantage is that it will make https more pervasive and hopefully people will start thinking https and forgetting all about http. A more secure internet is in all of our best interests.

Anyway, I'd like to (slowly) run this across the planet. I've discussed this on the US Slack channel and have performed the actions on the United States already. I've addressed many questions and have heard no strong objections. I'm seeking feedback from the larger community now before proceeding.


The Slack conversation is available, but has died down and the transcript is available at the wiki page mentioned above.

The diary entry with some more conversation is at the bot's page: https://www.openstreetmap.org/user/b-jazz-bot/diary/47743

The source code is available on GitLab for review: https://gitlab.com/b-jazz/https_all_the_things

Example changeset for a run over the "9yfd" geohash: https://www.openstreetmap.org/changeset/67454775

I welcome your input.



_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

Stephan Knauss
Hi,

Please be aware that protocol independent URLs do not mean that http is used. The client will simply continue using the protocol it used before.

Real need for that is quite limited. So in most cases they are better written as https.

But it then needs to be changed where the URL is used and not on the provider end.

Stephan


On February 22, 2019 8:02:20 AM GMT+01:00, Bryce Jasmer <[hidden email]> wrote:
I have written a script that will search for OSM objects that have a website tag that explicitly states "http://..." or implicitly uses http by leaving of the protocol specification. The script will then loop through all that it discovers and asks the http site if it will redirect me to the secure version of the website over the https protocol. If it does, I will update the database with the new value.

This has a couple of advantages. From now through the end of time, any user clicking on one of those links will be spared the time it takes to establish the connection, ask if there is a secure version of the site, and tear down the connection. It's on the order of 10-200 ms to do, but over the life of the link and the number of objects that are clicked and the population, this could save centuries of time :-)

Another advantage is that it will make https more pervasive and hopefully people will start thinking https and forgetting all about http. A more secure internet is in all of our best interests.

Anyway, I'd like to (slowly) run this across the planet. I've discussed this on the US Slack channel and have performed the actions on the United States already. I've addressed many questions and have heard no strong objections. I'm seeking feedback from the larger community now before proceeding.


The Slack conversation is available, but has died down and the transcript is available at the wiki page mentioned above.

The diary entry with some more conversation is at the bot's page: https://www.openstreetmap.org/user/b-jazz-bot/diary/47743

The source code is available on GitLab for review: https://gitlab.com/b-jazz/https_all_the_things

Example changeset for a run over the "9yfd" geohash: https://www.openstreetmap.org/changeset/67454775

I welcome your input.



_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

James-2
In reply to this post by Bryce Jasmer
soooo basically you copied this? https://www.eff.org/https-everywhere

On Fri., Feb. 22, 2019, 2:05 a.m. Bryce Jasmer, <[hidden email]> wrote:
I have written a script that will search for OSM objects that have a website tag that explicitly states "http://..." or implicitly uses http by leaving of the protocol specification. The script will then loop through all that it discovers and asks the http site if it will redirect me to the secure version of the website over the https protocol. If it does, I will update the database with the new value.

This has a couple of advantages. From now through the end of time, any user clicking on one of those links will be spared the time it takes to establish the connection, ask if there is a secure version of the site, and tear down the connection. It's on the order of 10-200 ms to do, but over the life of the link and the number of objects that are clicked and the population, this could save centuries of time :-)

Another advantage is that it will make https more pervasive and hopefully people will start thinking https and forgetting all about http. A more secure internet is in all of our best interests.

Anyway, I'd like to (slowly) run this across the planet. I've discussed this on the US Slack channel and have performed the actions on the United States already. I've addressed many questions and have heard no strong objections. I'm seeking feedback from the larger community now before proceeding.


The Slack conversation is available, but has died down and the transcript is available at the wiki page mentioned above.

The diary entry with some more conversation is at the bot's page: https://www.openstreetmap.org/user/b-jazz-bot/diary/47743

The source code is available on GitLab for review: https://gitlab.com/b-jazz/https_all_the_things

Example changeset for a run over the "9yfd" geohash: https://www.openstreetmap.org/changeset/67454775

I welcome your input.


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

Bryce Jasmer
In reply to this post by Stephan Knauss
Good point Stephan about protocol-less urls being left to the "browser" using the same protocol as it is currently using. But I think my approach is pretty sound in that I'll only update the value if there is a redirect from http to https. I did a sample of a dozen websites that don't redirect and tried out the https version of their site. 100% of them were broken. So I can't assume https, but trying http and looking for a published redirect seems pretty sensible to me.
 Thanks for the feedback.


On Fri, Feb 22, 2019 at 12:55 AM Stephan Knauss <[hidden email]> wrote:
Hi,

Please be aware that protocol independent URLs do not mean that http is used. The client will simply continue using the protocol it used before.

Real need for that is quite limited. So in most cases they are better written as https.

But it then needs to be changed where the URL is used and not on the provider end.

Stephan


On February 22, 2019 8:02:20 AM GMT+01:00, Bryce Jasmer <[hidden email]> wrote:
I have written a script that will search for OSM objects that have a website tag that explicitly states "http://..." or implicitly uses http by leaving of the protocol specification. The script will then loop through all that it discovers and asks the http site if it will redirect me to the secure version of the website over the https protocol. If it does, I will update the database with the new value.

This has a couple of advantages. From now through the end of time, any user clicking on one of those links will be spared the time it takes to establish the connection, ask if there is a secure version of the site, and tear down the connection. It's on the order of 10-200 ms to do, but over the life of the link and the number of objects that are clicked and the population, this could save centuries of time :-)

Another advantage is that it will make https more pervasive and hopefully people will start thinking https and forgetting all about http. A more secure internet is in all of our best interests.

Anyway, I'd like to (slowly) run this across the planet. I've discussed this on the US Slack channel and have performed the actions on the United States already. I've addressed many questions and have heard no strong objections. I'm seeking feedback from the larger community now before proceeding.


The Slack conversation is available, but has died down and the transcript is available at the wiki page mentioned above.

The diary entry with some more conversation is at the bot's page: https://www.openstreetmap.org/user/b-jazz-bot/diary/47743

The source code is available on GitLab for review: https://gitlab.com/b-jazz/https_all_the_things

Example changeset for a run over the "9yfd" geohash: https://www.openstreetmap.org/changeset/67454775

I welcome your input.



_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

Jmapb
In reply to this post by Bryce Jasmer
On 2/22/2019 2:02 AM, Bryce Jasmer wrote:
I have written a script that will search for OSM objects that have a website tag that explicitly states "http://..." or implicitly uses http by leaving of the protocol specification. The script will then loop through all that it discovers and asks the http site if it will redirect me to the secure version of the website over the https protocol. If it does, I will update the database with the new value.

This has a couple of advantages. From now through the end of time, any user clicking on one of those links will be spared the time it takes to establish the connection, ask if there is a secure version of the site, and tear down the connection. It's on the order of 10-200 ms to do, but over the life of the link and the number of objects that are clicked and the population, this could save centuries of time :-)

Another advantage is that it will make https more pervasive and hopefully people will start thinking https and forgetting all about http. A more secure internet is in all of our best interests.

Anyway, I'd like to (slowly) run this across the planet. I've discussed this on the US Slack channel and have performed the actions on the United States already. I've addressed many questions and have heard no strong objections. I'm seeking feedback from the larger community now before proceeding.


The Slack conversation is available, but has died down and the transcript is available at the wiki page mentioned above.

The diary entry with some more conversation is at the bot's page: https://www.openstreetmap.org/user/b-jazz-bot/diary/47743

The source code is available on GitLab for review: https://gitlab.com/b-jazz/https_all_the_things

Example changeset for a run over the "9yfd" geohash: https://www.openstreetmap.org/changeset/67454775

I welcome your input.

Hi Bryce -- I've been observing these automated changes around NYC. I'd like to humbly request you run these sorts of large projects by the Talk-US mailing list before implementation, since there are many mappers (dozens of us!) who don't choose to spend time on Slack. (Apologies if you did post and I missed it -- that's bound to happen sometimes.)

Personally I've been updating these tags to https manually as I come across them (sometimes prompted by Keepright), IF I can verify that the business (etc) in question is still a going concern and still located in the same place.

IMO the value of an automated edit when there's already a redirect in place is minimal enough that I don't think it justifies bumping the version and modification date. Just my opinion.

(Also, why are you adding a trailing slash to everything?)

Thanks for posting your code -- I'm contemplating an automated import of my own and I've been meaning to browse some modern bot code to get me started.

Jason


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

Mike N.
On 2/22/2019 3:36 PM, Jmapb wrote:
> IMO the value of an automated edit when there's already a redirect in
> place is minimal enough that I don't think it justifies bumping the
> version and modification date. Just my opinion.

   The value of the automated edit is that there is a small improvement
in security.   Assuming that someone ever clicks on a link in our data,
it is more secure to go directly to the HTTPS site rather than start
with the HTTP site.

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

Jmapb
On 2/22/2019 3:48 PM, Mike N wrote:

> On 2/22/2019 3:36 PM, Jmapb wrote:
>> IMO the value of an automated edit when there's already a redirect in
>> place is minimal enough that I don't think it justifies bumping the
>> version and modification date. Just my opinion.
>
>   The value of the automated edit is that there is a small improvement
> in security.   Assuming that someone ever clicks on a link in our
> data, it is more secure to go directly to the HTTPS site rather than
> start with the HTTP site.

True, that's exactly why I update these when I find them. I don't think
it warrants an automated edit, but there's room on the map for those who
do ;)

J


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

Mateusz Konieczny-3
In reply to this post by Bryce Jasmer
Feb 22, 2019, 8:02 AM by [hidden email]:
I have written a script that will search for OSM objects that have a website tag that explicitly states "http://..." or implicitly uses http by leaving of the protocol specification. The script will then loop through all that it discovers and asks the http site if it will redirect me to the secure version of the website over the https protocol. If it does, I will update the database with the new value.
no edit will be made, right?

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

Frederik Ramm
Hi,

On 26.02.19 12:47, Mateusz Konieczny wrote:
> So when http://domainname.com redirects to
> https://some-other-domainname.com <http://domainname.com>
> no edit will be made, right?

The logic for this appears to be here

https://gitlab.com/b-jazz/https_all_the_things/blob/master/src/httpsosm.py#L132-137

which reads:

if any((website.replace('http://', 'https://', 1) == new_location,
                       website.replace('http://', 'https://', 1) + '/'
== new_location,                             website.replace('http://',
'https://www.', 1) == new_location,
website.replace('http://', 'https://www.', 1) + '/' == new_location,
                         website.replace('http://www.', 'https://', 1)
== new_location,
website.replace('http://www.', 'https://', 1) + '/' == new_location)):
                       element['tags']['website'] = new_location

Bye
Frederik

--
Frederik Ramm  ##  eMail [hidden email]  ##  N49°00'09" E008°23'33"

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

Bryce Jasmer
Correct. No change will be made on anything other than the most straightforward of redirects. So even http://example.com -> https://example.com/home.aspx will be ignored. 

On Tue, Feb 26, 2019 at 4:23 AM Frederik Ramm <[hidden email]> wrote:
Hi,

On 26.02.19 12:47, Mateusz Konieczny wrote:
> So when http://domainname.com redirects to
> https://some-other-domainname.com <http://domainname.com>
> no edit will be made, right?

The logic for this appears to be here

https://gitlab.com/b-jazz/https_all_the_things/blob/master/src/httpsosm.py#L132-137

which reads:

if any((website.replace('http://', 'https://', 1) == new_location,
                       website.replace('http://', 'https://', 1) + '/'
== new_location,                             website.replace('http://',
'https://www.', 1) == new_location,
website.replace('http://', 'https://www.', 1) + '/' == new_location,
                         website.replace('http://www.', 'https://', 1)
== new_location,
website.replace('http://www.', 'https://', 1) + '/' == new_location)):
                       element['tags']['website'] = new_location

Bye
Frederik

--
Frederik Ramm  ##  eMail [hidden email]  ##  N49°00'09" E008°23'33"

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

Mateusz Konieczny-3

In that case this mechanical edit makes sense for me (as long as edits
will not create enormous bounding boxes due to grouping edits across country in one edit)

Feb 26, 2019, 1:34 PM by [hidden email]:
Correct. No change will be made on anything other than the most straightforward of redirects. So even http://example.com -> https://example.com/home.aspx will be ignored. 

On Tue, Feb 26, 2019 at 4:23 AM Frederik Ramm <[hidden email]> wrote:
Hi,

On 26.02.19 12:47, Mateusz Konieczny wrote:
> So when http://domainname.com redirects to
> no edit will be made, right?

The logic for this appears to be here


which reads:

if any((website.replace('http://', 'https://', 1) == new_location,
                       website.replace('http://', 'https://', 1) + '/'
== new_location,                             website.replace('http://',
'https://www.', 1) == new_location,
website.replace('http://', 'https://www.', 1) + '/' == new_location,
                         website.replace('http://www.', 'https://', 1)
== new_location,
website.replace('http://www.', 'https://', 1) + '/' == new_location)):
                       element['tags']['website'] = new_location

Bye
Frederik

--
Frederik Ramm  ##  eMail [hidden email]  ##  N49°00'09" E008°23'33"

_______________________________________________
talk mailing list


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

Bryce Jasmer
How would you feel about bounding boxes that cross country borders but are 3 geohash digits or smaller? (Sorry I cant give you an example at the moment, the power has been out so I can’t access tools on my computer.) I’m not sure what your definition of enormous is and what would be an acceptable size. 

On Tue, Feb 26, 2019 at 4:39 AM Mateusz Konieczny <[hidden email]> wrote:

In that case this mechanical edit makes sense for me (as long as edits
will not create enormous bounding boxes due to grouping edits across country in one edit)

Feb 26, 2019, 1:34 PM by [hidden email]:
Correct. No change will be made on anything other than the most straightforward of redirects. So even http://example.com -> https://example.com/home.aspx will be ignored. 

On Tue, Feb 26, 2019 at 4:23 AM Frederik Ramm <[hidden email]> wrote:
Hi,

On 26.02.19 12:47, Mateusz Konieczny wrote:
> So when http://domainname.com redirects to
> no edit will be made, right?

The logic for this appears to be here


which reads:

if any((website.replace('http://', 'https://', 1) == new_location,
                       website.replace('http://', 'https://', 1) + '/'
== new_location,                             website.replace('http://',
'https://www.', 1) == new_location,
website.replace('http://', 'https://www.', 1) + '/' == new_location,
                         website.replace('http://www.', 'https://', 1)
== new_location,
website.replace('http://www.', 'https://', 1) + '/' == new_location)):
                       element['tags']['website'] = new_location

Bye
Frederik

--
Frederik Ramm  ##  eMail [hidden email]  ##  N49°00'09" E008°23'33"

_______________________________________________
talk mailing list

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

Mateusz Konieczny-3
Crossing country border is OK for me. Problem is when one edit object is in say Moscow and second
just across Bering Strait resulting in edit bounding box going across entire continent.

In my bots I use 0.1 degrees as max size of bounding box in both latitute and longuitude,
except cases where edited objects are larger.


Feb 26, 2019, 1:48 PM by [hidden email]:
How would you feel about bounding boxes that cross country borders but are 3 geohash digits or smaller? (Sorry I cant give you an example at the moment, the power has been out so I can’t access tools on my computer.) I’m not sure what your definition of enormous is and what would be an acceptable size. 

On Tue, Feb 26, 2019 at 4:39 AM Mateusz Konieczny <[hidden email]> wrote:

In that case this mechanical edit makes sense for me (as long as edits
will not create enormous bounding boxes due to grouping edits across country in one edit)

Feb 26, 2019, 1:34 PM by [hidden email]:
Correct. No change will be made on anything other than the most straightforward of redirects. So even http://example.com -> https://example.com/home.aspx will be ignored. 

On Tue, Feb 26, 2019 at 4:23 AM Frederik Ramm <[hidden email]> wrote:
Hi,

On 26.02.19 12:47, Mateusz Konieczny wrote:
> So when http://domainname.com redirects to
> no edit will be made, right?

The logic for this appears to be here


which reads:

if any((website.replace('http://', 'https://', 1) == new_location,
                       website.replace('http://', 'https://', 1) + '/'
== new_location,                             website.replace('http://',
'https://www.', 1) == new_location,
website.replace('http://', 'https://www.', 1) + '/' == new_location,
                         website.replace('http://www.', 'https://', 1)
== new_location,
website.replace('http://www.', 'https://', 1) + '/' == new_location)):
                       element['tags']['website'] = new_location

Bye
Frederik

--
Frederik Ramm  ##  eMail [hidden email]  ##  N49°00'09" E008°23'33"

_______________________________________________
talk mailing list

_______________________________________________
talk mailing list


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

Andy Townsend
In reply to this post by Bryce Jasmer
On 26/02/2019 12:34, Bryce Jasmer wrote:
> Correct. No change will be made on anything other than the most
> straightforward of redirects. So even http://example.com ->
> https://example.com/home.aspx will be ignored.

What about certificate checking?  Suppose someone primarily uses http://
for accessing their server, but has either a self-signed certificate on
https:// or an untrusted / expired one (perhaps they were testing).  
Presumably in that case you wouldn't change http:// to https:// ?

Best Regards,

Andy


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

Joseph Reeves
This certificate question from Andy is a good one, and is the final reason I'm emailing to say I would vote against this proposed edit:
  1. I can't see the security risk you're trying to protect against. We are looking at applications that use OSM data and will refer users to third party websites; what is the risk of a malicious user MiTM'ing a http request to a restaurant website (for example) and sending me to location other than the https version of the site? What web clients are you expecting this applies to?
  2. I can see in the comments of your diary entry that you were told about HSTS recently. I'm not trying to be offensive, but that shows you're not a HTTPS / web security expert. Do you really think you're the person to be making world wide automatic changes to the database? As an aside, HSTS is interesting here because the website operator is saying "only use this domain over https", but at that point, we don't need to make changes to the database because the web client should be aware of the HSTS preload list; the protocol listed in the referrer is not relevant.
  3. Again, are you checking https certificates? Do you know that the https site actually works?
  4. Are you checking the redirect code? Do you differentiate between temporary and permanent redirects?
  5. Are redirects even that bad? If I was to set up some careful redirects and have them ignored by a bot that thinks it knows better, I may be a little annoyed. What about geographic redirects? http://example.com becomes https://de.example.com, for example.
  6. A different, but related issue: You say you "abhor www", but does that mean you should be making changes based on this? What about the people that like www. ? www. and the bare domain can be different hosts, so what about the small number of cases in which people host a different site on the bare domain? I notice your own domain resolves a different IP for the bare domain and the www subdomain.
I can see that you want to promote https adoption, but I can't see that the OSM database is the place to do it. In the end, the website operator is responsible for deciding upon transport security, or not, and in how they publicise their sites; working with site operators, I think there is better work to be done encouraging https adoption outside of OSM, or more advanced topics such as HSTS. I also think you could explore the applications that use OSM data, and determine if they're using resources such as the HSTS preload list.

I don't think there has been enough consideration of some of the issues here, and I think an automated bot edit would create a lot of noise without any obvious improvements.

Cheers, Joseph



On Tue, 26 Feb 2019 at 13:14, Andy Townsend <[hidden email]> wrote:
On 26/02/2019 12:34, Bryce Jasmer wrote:
> Correct. No change will be made on anything other than the most
> straightforward of redirects. So even http://example.com ->
> https://example.com/home.aspx will be ignored.

What about certificate checking?  Suppose someone primarily uses http://
for accessing their server, but has either a self-signed certificate on
https:// or an untrusted / expired one (perhaps they were testing). 
Presumably in that case you wouldn't change http:// to https:// ?

Best Regards,

Andy


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

Frederik Ramm
Hi,

when I first read about this planned edit, I was critical too; I
thought, "ah, another eager youngster wanting to make the world a more
secure place by telling everyone else how they ought to conduct their
business".

But if I haven't totally misunderstood this, then the proposal will only
replace a http:// by a https:// pointer if the site operator himself has
added that redirect in their web server configuration.

So yes, the SSL certificate might be invalid or self-signed, but if the
operator has configured his server to redirect everyone to that broken
certificate then visiting the site with http will not improve your
existence in any way.

Had this suggested edit been "I'll simply try port 443 and if that's
open I'll re-write the http URL to https" then it would of course not be
acceptable.

But I struggle to find any problems with the suggestion, other than my
general reservation against any automated edit - it will make the object
"look fresh" when indeed it hasn't been touched. In the worst case, an
object might have a wrong web site URL, that points to the web site of
something completely different, and this bot would happily edit the web
site, still pointing to something completely different. But it wouldn't
exactly worsen the situtation...

Bye
Frederik

--
Frederik Ramm  ##  eMail [hidden email]  ##  N49°00'09" E008°23'33"

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

Bryce Jasmer
In reply to this post by Andy Townsend
In that situation, the admin wouldn’t redirect all of their traffic to their test site with a potentially broken cert. The bot will only modify objects where the admin is specifically redirecting traffic already. It makes no assumptions. The scope is very limited for this exact reason. It will NOT guess that just because something is listening on 443 that it should make changes. 

On Tue, Feb 26, 2019 at 5:12 AM Andy Townsend <[hidden email]> wrote:
On 26/02/2019 12:34, Bryce Jasmer wrote:
> Correct. No change will be made on anything other than the most
> straightforward of redirects. So even http://example.com ->
> https://example.com/home.aspx will be ignored.

What about certificate checking?  Suppose someone primarily uses http://
for accessing their server, but has either a self-signed certificate on
https:// or an untrusted / expired one (perhaps they were testing). 
Presumably in that case you wouldn't change http:// to https:// ?

Best Regards,

Andy


_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

ebel
In reply to this post by Joseph Reeves
On 26/02/2019 14:45, Joseph Reeves wrote:
> As an aside, HSTS is interesting here because the website operator is
> saying "only use this domain over https", but at that point, we don't
> need to make changes to the database because the web client should be
> aware of the HSTS preload list; the protocol listed in the referrer
> is not relevant.

I don't think we can rely totally on HSTS. I'm sure not all sites are on
HSTS preload lists. I think OSM has more "website=http://*" tags (965k)¹
than Firefox² & Chrome³ have in their HSTS preload lists...

[1] https://taginfo.openstreetmap.org/keys/website#values

[2]
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security#Preloading_Strict_Transport_Security
https://hg.mozilla.org/mozilla-central/raw-file/tip/security/manager/ssl/nsSTSPreloadList.inc

[3]
https://www.chromium.org/hsts
https://cs.chromium.org/codesearch/f/chromium/src/net/http/transport_security_state_static.json?cl=5b2537d89ea5994d27bba5735961b0be1095c54c

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

Bryce Jasmer
The HSTS discussion is completely orthogonal to what the stated goal is and any further discussion on it is really just muddying the waters. HSTS comes into play after the user is already visiting over https. 

If I’m mistaken, please help me understand. 

On Tue, Feb 26, 2019 at 6:30 AM Rory McCann <[hidden email]> wrote:
On 26/02/2019 14:45, Joseph Reeves wrote:
> As an aside, HSTS is interesting here because the website operator is
> saying "only use this domain over https", but at that point, we don't
> need to make changes to the database because the web client should be
> aware of the HSTS preload list; the protocol listed in the referrer
> is not relevant.

I don't think we can rely totally on HSTS. I'm sure not all sites are on
HSTS preload lists. I think OSM has more "website=http://*" tags (965k)¹
than Firefox² & Chrome³ have in their HSTS preload lists...

[1] https://taginfo.openstreetmap.org/keys/website#values

[2]
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security#Preloading_Strict_Transport_Security
https://hg.mozilla.org/mozilla-central/raw-file/tip/security/manager/ssl/nsSTSPreloadList.inc

[3]
https://www.chromium.org/hsts
https://cs.chromium.org/codesearch/f/chromium/src/net/http/transport_security_state_static.json?cl=5b2537d89ea5994d27bba5735961b0be1095c54c

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS all the Things (Automated Edit)

Joseph Reeves
In reply to this post by ebel
Hi Rory,

Sure, so my point is: If someone wants to encourage https adoption in the wider world, the OSM database is not the place to do it. Security mechanisms exist for website operators to implement if they so desire, and they may need help making the most appropriate decisions.

Cheers, Joseph

On Tue, 26 Feb 2019 at 14:30, Rory McCann <[hidden email]> wrote:
On 26/02/2019 14:45, Joseph Reeves wrote:
> As an aside, HSTS is interesting here because the website operator is
> saying "only use this domain over https", but at that point, we don't
> need to make changes to the database because the web client should be
> aware of the HSTS preload list; the protocol listed in the referrer
> is not relevant.

I don't think we can rely totally on HSTS. I'm sure not all sites are on
HSTS preload lists. I think OSM has more "website=http://*" tags (965k)¹
than Firefox² & Chrome³ have in their HSTS preload lists...

[1] https://taginfo.openstreetmap.org/keys/website#values

[2]
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security#Preloading_Strict_Transport_Security
https://hg.mozilla.org/mozilla-central/raw-file/tip/security/manager/ssl/nsSTSPreloadList.inc

[3]
https://www.chromium.org/hsts
https://cs.chromium.org/codesearch/f/chromium/src/net/http/transport_security_state_static.json?cl=5b2537d89ea5994d27bba5735961b0be1095c54c

_______________________________________________
talk mailing list
[hidden email]
https://lists.openstreetmap.org/listinfo/talk
123