Alex Deering

I havent been able to figure out how to turn on logs for faraday

So increased the timeout to a minute on Searchkick and that got me a different error. Tried to see if I could find a solution (and will keep looking), but figured Id update here in case you know whats going on:

Elasticsearch::Transport::Transport::Errors::ServiceUnavailable ([503] {"error":{"root_cause":[{"type":"master_not_discovered_exception","reason":null}],"type":"master_not_discovered_exception","reason":null},"status":503})

Hmmm, no idea whats causing it. I checked the curl from the server as well and got the same. And using searchkick with is configured to use ENV['ELASTICSEARCH_URL'] which is set to the same thing. Everything works in development environment as expected and the only fault im getting is the timed out....Any suggestions on what to try to get more info?

It was setup with Hatchbox, here is the provision log:

-----> Connecting to living-recipe-elasticsearch (138.68.24.137 port 22) as root

Uploaded /etc/sudoers.d/deploy

Executing 'chmod 440 /etc/sudoers.d/deploy'

Uploaded .hatch/action-272036.sh

Executing 'set -euxo pipefail; timeout 900 bash .hatch/action-272036.sh  2>&1'
+ timeout 900 bash .hatch/action-272036.sh

-----> Disabling password authentication for SSH

-----> Installing initial dependencies

-----> Adding repositories
Warning: apt-key output should not be parsed (stdout is not a terminal)
OK

-----> Updating packages

-----> Upgrading packages

-----> Installing dependencies
ca-certificates ufw default-jre elasticsearch

-----> Setting up firewall rules
Skipping adding existing rule
Skipping adding existing rule (v6)
Rule added
Skipping adding existing rule
Firewall is active and enabled on system startup

-----> Post-install configuration
Synchronizing state of elasticsearch.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable elasticsearch

-----> Done!

Uploaded /etc/apt/apt.conf.d/10periodic

Uploaded /etc/update-motd.d/60-hatchbox

Executing 'chmod +x /etc/update-motd.d/60-hatchbox'

And the ENV['ELASTICSEARCH_URL'] is pointing to the Private IP of the elasticsearch server.

Got another hatchbox issue trying to figure out. With elasticsearch now, I have a dedicated cluster that only runs elasticsearch. In the ENV there is the elasticsearch_url and ive SSH into the elasticsearch cluster and checked the connection and everything comes back good. But when I try to reindex thru the console i get the following:

rake aborted!
Faraday::Error::TimeoutError: request timed out
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/typhoeus-1.3.1/lib/typhoeus/adapters/faraday.rb:100:in `block in request'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/typhoeus-1.3.1/lib/typhoeus/request/callbacks.rb:146:in `block in execute_callbacks'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/typhoeus-1.3.1/lib/typhoeus/request/callbacks.rb:145:in `each'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/typhoeus-1.3.1/lib/typhoeus/request/callbacks.rb:145:in `execute_callbacks'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/typhoeus-1.3.1/lib/typhoeus/request/operations.rb:35:in `finish'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/typhoeus-1.3.1/lib/typhoeus/easy_factory.rb:164:in `block in set_callback'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/ethon-0.12.0/lib/ethon/easy/response_callbacks.rb:68:in `block in complete'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/ethon-0.12.0/lib/ethon/easy/response_callbacks.rb:68:in `each'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/ethon-0.12.0/lib/ethon/easy/response_callbacks.rb:68:in `complete'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/ethon-0.12.0/lib/ethon/easy/operations.rb:33:in `perform'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/typhoeus-1.3.1/lib/typhoeus/request/operations.rb:16:in `run'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/typhoeus-1.3.1/lib/typhoeus/request/cacheable.rb:18:in `run'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/typhoeus-1.3.1/lib/typhoeus/request/block_connection.rb:31:in `run'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/typhoeus-1.3.1/lib/typhoeus/request/stubbable.rb:25:in `run'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/typhoeus-1.3.1/lib/typhoeus/request/before.rb:26:in `run'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/typhoeus-1.3.1/lib/typhoeus/adapters/faraday.rb:82:in `perform_request'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/typhoeus-1.3.1/lib/typhoeus/adapters/faraday.rb:72:in `call'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/searchkick-4.1.1/lib/searchkick/middleware.rb:9:in `call'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/faraday-0.17.1/lib/faraday/rack_builder.rb:143:in `build_response'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/faraday-0.17.1/lib/faraday/connection.rb:387:in `run_request'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/elasticsearch-transport-7.4.0/lib/elasticsearch/transport/transport/http/faraday.rb:27:in `block in perform_request'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/elasticsearch-transport-7.4.0/lib/elasticsearch/transport/transport/base.rb:274:in `perform_request'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/elasticsearch-transport-7.4.0/lib/elasticsearch/transport/transport/http/faraday.rb:24:in `perform_request'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/elasticsearch-transport-7.4.0/lib/elasticsearch/transport/client.rb:152:in `perform_request'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/elasticsearch-api-7.4.0/lib/elasticsearch/api/namespace/common.rb:25:in `perform_request'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/elasticsearch-api-7.4.0/lib/elasticsearch/api/actions/indices/get_alias.rb:36:in `get_alias'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/searchkick-4.1.1/lib/searchkick/index.rb:106:in `all_indices'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/searchkick-4.1.1/lib/searchkick/index.rb:119:in `clean_indices'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/searchkick-4.1.1/lib/searchkick/index.rb:274:in `reindex_scope'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/searchkick-4.1.1/lib/searchkick/index.rb:200:in `reindex'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/searchkick-4.1.1/lib/searchkick/model.rb:60:in `searchkick_reindex'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/searchkick-4.1.1/lib/tasks/searchkick.rake:7:in `block (2 levels) in <main>'
/home/deploy/living-recipe/shared/bundle/ruby/2.6.0/gems/rake-13.0.1/exe/rake:27:in `<top (required)>'
/home/deploy/.rbenv/versions/2.6.5/bin/bundle:23:in `load'
/home/deploy/.rbenv/versions/2.6.5/bin/bundle:23:in `<main>'
Tasks: TOP => searchkick:reindex
(See full trace by running task with --trace)

Finally figured it out. The connection was there, just rails db:setup was failing, but rails db:schema:load did work.

Thanks again for all your help and quick responses

Found the problem with the storage, didnt have the full endpoint in the storage config for digitalocean. It deploys now and the index page loads but as soon as I navigate to a page that tries and queries the database i get an error like this:

[eabfc969-459b-45d4-9fdd-124d60bd1e02] ActiveRecord::StatementInvalid (PG::UndefinedTable: ERROR:  relation "equipment" does not exist
LINE 8:  WHERE a.attrelid = '"equipment"'::regclass
                            ^
):

Looks like the database isnt being created initially (which looking at the deploy script it just runs migrate, which my migrations unfortunatley got messed up. so need to run db:setup)
So I tried to SSH in and manually setup the database with the following:

bundle exec rails db:setup DISABLE_DATABASE_ENVIRONMENT_CHECK=1 SAFETY_ASSURED=1

and I get these errors:

FATAL:  database "postgres" does not exist
Couldn't create 'defaultdb' database. Please check your configuration.
rails aborted!
ActiveRecord::NoDatabaseError: FATAL:  database "postgres" does not exist

My database.yml for production has this production:
url: <%= ENV['DATABASE_URL'] %> and I have the connection string from the digitaloceans postgres settings:
postgresql://username:password@db-postgresql-sfo2-71324-do-user-6600757-0.db.ondigitalocean.com:25060/dbname?sslmode=require

But it doesnt seem to want to connect

This is the error:

[ E 2019-12-31 21:44:09.9456 15620/Tf age/Cor/App/Implementation.cpp:221 ]: Could not spawn process for application /home/deploy/livingrecipe/current: The application encountered the following error: Cannot load `Rails.config.active_storage.service`:
expected :endpoint to be a HTTP or HTTPS endpoint (ArgumentError)
  Error ID: 328905b0
  Error details saved to: /tmp/passenger-error-8ltipm.htm

Well keep getting further. Now Hatchbox deploys the app successfully but it has this comment:

-----> Restarting server
There are no Phusion Passenger-served applications running whose paths begin with '/home/deploy/livingrecipe/current'.
Branch master (at 8bc3d26710784ffc568517f70ce380a0b3781e32) deployed as release 20191231212424 by deploy

That was it

Well got past the key stuff, now I have some issues with yarn. Get this error when trying to install:

ERROR in ./node_modules/select2/dist/css/select2.css
Module build failed (from ./node_modules/mini-css-extract-plugin/dist/loader.js):
ModuleBuildError: Module build failed (from ./node_modules/postcss-loader/src/index.js):
Error: Cannot find module '@tailwindcss/custom-forms'
Require stack:
- /home/deploy/livingrecipe/releases/20191231202322/app/javascript/stylesheets/tailwind.config.js
- /home/deploy/livingrecipe/releases/20191231202322/node_modules/tailwindcss/lib/index.js
- /home/deploy/livingrecipe/releases/20191231202322/postcss.config.js
- /home/deploy/livingrecipe/releases/20191231202322/node_modules/import-fresh/index.js
- /home/deploy/livingrecipe/releases/20191231202322/node_modules/cosmiconfig/dist/loaders.js
- /home/deploy/livingrecipe/releases/20191231202322/node_modules/cosmiconfig/dist/createExplorer.js
- /home/deploy/livingrecipe/releases/20191231202322/node_modules/cosmiconfig/dist/index.js
- /home/deploy/livingrecipe/releases/20191231202322/node_modules/cssnano/dist/index.js
- /home/deploy/livingrecipe/releases/20191231202322/node_modules/optimize-css-assets-webpack-plugin/src/index.js
- /home/deploy/livingrecipe/releases/20191231202322/node_modules/@rails/webpacker/package/environments/production.js
- /home/deploy/livingrecipe/releases/20191231202322/node_modules/@rails/webpacker/package/index.js
- /home/deploy/livingrecipe/releases/20191231202322/config/webpack/environment.js
- /home/deploy/livingrecipe/releases/20191231202322/config/webpack/production.js
- /home/deploy/livingrecipe/releases/20191231202322/node_modules/webpack-cli/bin/utils/convert-argv.js
- /home/deploy/livingrecipe/releases/20191231202322/node_modules/webpack-cli/bin/cli.js
- /home/deploy/livingrecipe/releases/20191231202322/node_modules/webpack/bin/webpack.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:797:15)
    at Function.Module._load (internal/modules/cjs/loader.js:690:27)
    at Module.require (internal/modules/cjs/loader.js:852:19)
    at require (/home/deploy/livingrecipe/releases/20191231202322/node_modules/v8-compile-cache/v8-compile-cache.js:161:20)
    at Object. (/home/deploy/livingrecipe/releases/20191231202322/app/javascript/stylesheets/tailwind.config.js:62:5)
    at Module._compile (/home/deploy/livingrecipe/releases/20191231202322/node_modules/v8-compile-cache/v8-compile-cache.js:192:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:995:10)
    at Module.load (internal/modules/cjs/loader.js:815:32)
    at Function.Module._load (internal/modules/cjs/loader.js:727:14)
    at Module.require (internal/modules/cjs/loader.js:852:19)
    at require (/home/deploy/livingrecipe/releases/20191231202322/node_modules/v8-compile-cache/v8-compile-cache.js:161:20)
    at /home/deploy/livingrecipe/releases/20191231202322/node_modules/tailwindcss/lib/index.js:67:122
    at /home/deploy/livingrecipe/releases/20191231202322/node_modules/tailwindcss/lib/processTailwindFeatures.js:32:20
    at LazyResult.run (/home/deploy/livingrecipe/releases/20191231202322/node_modules/postcss/lib/lazy-result.js:295:14)
    at LazyResult.asyncTick (/home/deploy/livingrecipe/releases/20191231202322/node_modules/postcss/lib/lazy-result.js:208:26)
    at LazyResult.asyncTick (/home/deploy/livingrecipe/releases/20191231202322/node_modules/postcss/lib/lazy-result.js:221:14)
    at /home/deploy/livingrecipe/releases/20191231202322/node_modules/postcss/lib/lazy-result.js:250:14
    at new Promise ()
    at LazyResult.async (/home/deploy/livingrecipe/releases/20191231202322/node_modules/postcss/lib/lazy-result.js:246:23)
    at LazyResult.then (/home/deploy/livingrecipe/releases/20191231202322/node_modules/postcss/lib/lazy-result.js:127:17)
    at /home/deploy/livingrecipe/releases/20191231202322/node_modules/postcss-loader/src/index.js:142:8
    at /home/deploy/livingrecipe/releases/20191231202322/node_modules/webpack/lib/NormalModule.js:316:20
    at /home/deploy/livingrecipe/releases/20191231202322/node_modules/loader-runner/lib/LoaderRunner.js:367:11
    at /home/deploy/livingrecipe/releases/20191231202322/node_modules/loader-runner/lib/LoaderRunner.js:233:18
    at context.callback (/home/deploy/livingrecipe/releases/20191231202322/node_modules/loader-runner/lib/LoaderRunner.js:111:13)
    at /home/deploy/livingrecipe/releases/20191231202322/node_modules/postcss-loader/src/index.js:208:9
 @ ./app/javascript/packs/application.js 8:283-320

Got it, thanks for the help with this

I did, and that is the secret_key_base in the credentials file correct? Still getting use to the new credentials setup

So ran into a precompile issue with some of my keys. I have all my keys in the production credentials file but its dont recognizing it. I changed the code to reference an ENV variable and added my keys that way thru hatchbox. Does hatchbox not utilize the production credentials file?

Ok got it, thanks

ok to make sure I got this right, in my database.yml file i have the following:

production:
<<: *default
database: defaultdb
username: ******
password: <%= ENV['LIVINGRECIPE_DATABASE_PASSWORD'] %>
url: <%= ENV['DATABASE_URL'] %>

Then in my credentials/production.yml file I have the following:

#database stuff
LIVINGRECIPE_DATABASE_PASSWORD: **************
DATABASE_URL: ************************************

Both of those taken from digitalocean. I used the private host address for the url as it is on the same data center as my clusters will be.

Im launching my app and using your Hatchbox service to help. I was planning on using digitaloceans managed database and wondering how that works with hatchbox and if I need to change anything in my production credentials to make that work

So this is less a question on how to set up integration with sendgrid or mailchimp, and more along the lines of how would you go about creating an admin setup to create dynamic newsletters without having to write them thru code and push an update.

Specifically what im looking to do is send out newsletters about which recipes are trending, popular, in season, etc... So its not just a text based email but will need to toss in queries and css formatting. So far Ive figured out canned emails like welcome emails but not sure how to go about setting up a system where i can dynamically create emails in the admin section.

Trying to set up digital ocean spaces in my development environment in order to test it out and make sure all the settings are correct before i push to production. But getting a Access blocked by CORS policy error for localhost domain. Tried to add it in the spaces settings but its not recognizing http://localhost:5000 as a valid domain. Is there no way to test this on local machine?

Also have a couple questions about CDN and File Listing

CDN, what is it and should it be used for a rails app. App is recipe database so users will be uploading recipe images and user avatars, will be delivering a lot of images.

File Listing: Should this be set to private or public?

Thanks for the help

ok so just downloaded the javascript file from the gem and did that and everything works...thanks