Check all links

[crandmck]

We need to run a comprehensive link-check on the site to identify broken links.

[doublemarked]

Hey Rand - I've given the links a pass. There are two types of problems, 404's and links to valid pages but with an invalid fragment identifier. I've stuck everything in the following spreadsheet:

https://docs.google.com/spreadsheets/d/1cXtr617d_FiK2dDernkWkrJ8tFZRXw3ptqWpyQe1zTw/edit#gid=1949899933

Give it a look at let me know if you have any questions. There are a lot of dupes in the first sheet from broken links in header/footer (feeds.xml, Command-line-reference.html). I believe there are only ~50 unique 404s. I've only processed the LB2 docs.

[crandmck]

Awesome! I'll get on it ASAP.

[crandmck]

Hi @doublemarked I ran thru that spreadsheet, and fixed all the links--well, I think so anyway. Many of the breakages were due to a couple broken links in the nav sidebar and in include templates, so that knocked out a bunch out.

If you would like, please run the link check again, and we'll see what's left! :-)

Thanks for your help!

[doublemarked]

@crandmck Updated the spreadsheet: https://docs.google.com/spreadsheets/d/1cXtr617d_FiK2dDernkWkrJ8tFZRXw3ptqWpyQe1zTw/edit#gid=487594605

From the looks of it, just a handful of 404s remain but a lot of identifiers need to be fixed.

[doublemarked]

Let me share with you how I'm generating these lists. I'm still happy to rerun things too, just let me know.

For the broken links I'm using the following tool: http://peacockmedia.software/mac/integrity-plus/

For the bad identifiers I hacked together the following (execute from within _site/doc/en/lb2):

1. Find all identifiers and map them into a list of valid urls w/ identifiers: for i in *.html; do export FILE=$i; perl -ne '/id="([^"]+?)"/ && print "$ENV{FILE}#$1\n"' $i; done > valid.txt
2. Find all links that reference other docs w/ identifiers and filter out valid ones: for i in *.html; do export FILE=$i; perl -ne '/href="\/doc\/en\/lb2\/([^"#]+?)#([^"]+?)"/ && !fgrep -l "$1#$2" valid.txt && print "$ENV{FILE}\t$1#$2\n"' $i; done > invalid.txt

[crandmck]

Awesome, thanks @doublemarked. I'll try to get to this over the weekend.

[crandmck]

Thanks again @doublemarked, I fixed all the links identified in the Google doc. And I fixed a few others I found myself. ...

Thanks alsofor providing how you did this--could be useful in the future. I didn't have a chance to run it myself, so if you want to do so again, that would be great. At this point, I hope that most of the broken links have been fixed!

[doublemarked]

@crandmck getting closer, but a few things remain. Spreadsheet updated w/ round 3 data!

[crandmck]

OK, done! Third time is charm, I hope... (fingers crossed)

[doublemarked]

The 404s from the crawler are unchanged. However, it looks like all the bad identifiers are fixed :)

[crandmck]

Ha ha, I missed that whole tab!

So, the broken link to feed.xml was on every page. Fixed that with an edit to _config.yml.

I fixed all the other links, except for:

• Links to "tag" pages, e.g. http://loopback.io/doc/en/lb2/tag_getting_started.html. This is already covered by Tags are broken #41. I hope to fix it soon, but it's not top priority. (I colored these green in the sheet).
• Links in example app pages, e.g. http://loopback.io/doc/en/lb2/Tutorial-access-control.html. These are copied into the repo from various example repos (e.g. https://github.com/strongloop/loopback-example-access-control). The whole list is in https://github.com/strongloop/loopback.io/blob/gh-pages/_data/repos.json. We need to fix these links in the individual repo README files. This is covered in Edit all example READMEs for proper formatting, etc. strongloop/loopback#2775, and unfortunately may have to wait a bit... unless folks in the community want to step up! I marked these red in the sheet.
• Links in data source connector READMEs. Same issue as above, but covered by Replace pages for connectors with READMEs #53. Marked as red in the sheet.
• A number of pages at the bottom of the sheet (which I marked in bold), that are not in the repo, e.g. http://loopback.io/doc/en/lb2/server/views/projects.ejs This may be some kind of artifact of the link checker?

Anyway, I'm going to go ahead and close this issue, because I think we're pretty close to being done. If there are still some broken links, please either reopen or open an new issue.

THANK YOU @doublemarked for your help!!

[doublemarked]

@crandmck ok all sounds good!

Just one response regarding,

A number of pages at the bottom of the sheet (which I marked in bold), that are not in the repo, e.g. http://loopback.io/doc/en/lb2/server/views/projects.ejs This may be some kind of artifact of the link checker?

Does not appear to be an artifact of the link checker. These are bad relative links to things that are probably supposed to be off-site or something. For example the projects.ejs one - it's caused by this link from readmes/loopback-example-access-control.md (included by Tutorial-access-control.md):

Sets the [`POST /projects` route to to render `projects.ejs` when credentials are valid](server/views/projects.ejs) and [renders `index.ejs`](https://github.com/strongloop/loopback-example-access-control/blob/master/server/views/index.ejs) when credentials are invalid

[crandmck]

These are bad relative links to things that are probably supposed to be off-site or something.

Aha! Thanks for clarification. This makes more sense.

This is caused by relative links in included READMEs (2nd bullet in my previous comment). So, we'll address those in each README. Basically: a relative link in a README to another file in the repo works fine in GitHub, but not when the document is used anywhere else (e.g. npm or loopback.io).