[HOWTO] Running XenForo in App Engine

Discussion in 'Other' started by xfrocks, Nov 16, 2013.

  1. xfrocks

    xfrocks XenForo rocks!
    Staff Member

    #1 , Nov 16, 2013
    Last edited: Mar 21, 2014

    The PHP Runtime for Google App Engine is currently in Preview and we have been able to make XenForo work perfectly in App Engine without any core file edits (one additional add-on is required). We will try to describe the steps clearly but before starting, you may want to get yourself comfortable with GAE:
    The GAE helper add-on is available for Webmaster here.

    Steps:
    And finally some benchmark results.
     
  2. xfrocks

    xfrocks XenForo rocks!
    Staff Member

    #2 xfrocks, Nov 16, 2013
    Last edited: Dec 1, 2013
    Step 1: Install XenForo and styles/add-ons on your local server

    Because running the installer for XenForo and styles/add-ons takes a lot of time on GAE (and time equals money). It is recommended to do all the heavy work on your machine first then upload the files + import the database into GAE later.

    You must have PHP, MySQL installed on your machine (of course!) and you will need to download an additional piece of software: the Google App Engine for PHP SDK (get it here).
    Also, you should go ahead and create a Cloud Console project, grab the project ID to use later.

    Step 1a: Setup GAE related configuration
    We will need to prepare 2 files in XenForo root directory: app.yaml (app configuration), php.ini (php configuration for the app).

    app.yaml

    Code:
    application: CLOUD_CONSOLE_PROJECT_ID
    version: 1
    runtime: php
    api_version: 1
    threadsafe: true
    
    handlers:
    - url: /js
      static_dir: js
    
    - url: /styles
      static_dir: styles
    
    - url: /install/install.css
      static_files: install/install.css
      upload: /install/install.css
    
    - url: /install/.*
      script: install/index.php
    
    - url: /admin.php
      script: admin.php
    
    - url: /css.php
      script: css.php
    
    - url: /deferred.php
      script: deferred.php
    
    - url: /fb_channel.php
      script: fb_channel.php
    
    - url: /payment_callback.php
      script: payment_callback.php
    
    - url: /rgba.php
      script: rgba.php
    
    - url: /.*
      script: index.php
    
    It is pretty straightfoward to understand
    • Find all files under "/js" and serve them as static files.
    • The same for files under "/styles".
    • One additional rule for the file "/install/install.css".
    • For all other URL that matches "/install/*", run "/install/install.php" to dynamically serve the request.
    • So on... For more information regarding app.yaml, read here.

    php.ini

    Code:
    mysqli.default_socket=/cloudsql/CLOUD_CONSOLE_PROJECT_ID:CLOUD_SQL_INSTANCE
    
    google_app_engine.enable_functions = "getmypid, phpinfo"
    You should notice a few things:
    • Because of the way mysqli connects to the database, we need to setup the socket in php.ini
    • CLOUD_CONSOLE_PROJECT_ID is the project ID of the Cloud Console project you created.
    • CLOUD_SQL_INSTANCE is the instance of Cloud SQL that you should create in your project.
    • And because GAE disable some built-in functions and XenForo happens to use a few of them, we have to enable them here (getmypid and phpinfo for now)
    • Actually, there are some functions that are permanently disabled in GAE and we have to patch them using the add-on later.

    Step 1b: Install XenForo 1.2.2
    Just follow the steps in the standard installation guide and you should be fine.

    Step 1c: Install [bd] Google App Engine Helper add-on
    Download the add-on in the first post and do these steps:
    1. Copy all files under "upload" directory into XenForo root directory
    2. Edit config.php, add these two lines
      PHP:
      $config['gae']['bucketName'] = 'CLOUD_STORAGE_BUCKET';
      bdGAE_Helper_Config::update($config);
    3. Where CLOUD_STORAGE_BUCKET is your Cloud Storage Bucket (more information here). `bucketName` allows you to specify a bucket to store avatars and attachments.
    4. Other configurations available are:
      1. `privateBucketName`: enter name of the bucket to store internal data (replace built-in internal_data directory). If you configure this, you should have the bucket for `bucketName` set to public access and the bucket for `privateBucketName` set to authorized access only. If you don't configure this, the add-on will use the bucket in `bucketName` for everything.
      2. `autoPatch`: determine whether the add-on should patch the files itself. If you wish to manually patch them for a small performance gain, you can do that and set this to `false` (default is `true`).
    5. The line `bdGAE_Helper_Config::update($config);` is required and must be included.
    Step 1d: Install other styles and add-ons as needed.
    Do it as per each style/add-on instruction. I would suggest install them, test to make sure everything work and disable all of them in AdminCP before deploying to GAE.
     
  3. xfrocks

    xfrocks XenForo rocks!
    Staff Member

    Step 2: Deploy to Google App Engine

    Step 2a: Set up Cloud SQL instance
    See step 6 here.

    Step 2b: Deploy
    See step 8 here.
     
  4. xfrocks

    xfrocks XenForo rocks!
    Staff Member

    #4 xfrocks, Dec 1, 2013
    Last edited: Dec 1, 2013
  5. xfrocks

    xfrocks XenForo rocks!
    Staff Member

    Reserved.
     
  6. Chojin

    Chojin New Member

    #6 Chojin, Dec 31, 2013
    Last edited: Jan 1, 2014
    Great module.
    Had to deal with 3 issues:
    • Serving image (data-internal) through Google Cloud Storage bucket which Google always says Access denied to image. I searched for a moment, it is not an ACL permission problem. I tested with a sample script and still have this error. So I changed in library/bdGAE/Helper/CloudStorage.php:
    Code:
           public static function getImageServingUrl($gsPath, $options)
            {
                    //return google\appengine\api\cloud_storage\CloudStorageTools::getImageServingUrl($gsPath, $options);
                    return substr_replace($gsPath, "http://commondatastorage.googleapis.com/", 0, 5);
            }
    
    • Serving image (data) through Google Cloud Storage bucket with public URL needs to be updated because the default one always asking for Google credentials. So I changed in library/bdGAE/Helper/Config.php:
    Code:
    // $config['externalDataUrl'] = sprintf('https://storage.cloud.google.com/%s', $config['gae']['bucketName']);
    $config['externalDataUrl'] = sprintf('http://commondatastorage.googleapis.com/%s', $config['gae']['bucketName']);
    
    • Cannot send mail. Something wrong with your code. For unknown reason when code reachs $message->addHeaderArray($headers) it does not continue (because $headers are empty). So I commented the code in library/bdGAE/Helper/Mail.php:
    Code:
    //                      if (!empty($headers))
    //                      {
    //                              $message->addHeaderArray($headers);
    //                      }
    
    Going to use it in prod.
    Also attachment support would be interesting ;)
     
    1. xfrocks

      xfrocks XenForo rocks!
      Staff Member

      xfrocks @Chojin Thanks for the post. I need to check again with the image URL, it uses GAE library, it should work...
      Regarding email, does it work after you commented that out?

      And what did you mean when you say "attachment support"?
       
  7. Chojin

    Chojin New Member

    Yup, email does work after commented that out.
    By "attachment support", I mean I cannot currently attach any file to a post.
     
    1. xfrocks

      xfrocks XenForo rocks!
      Staff Member

      xfrocks @Chojin Strange, please try at gae dot xfrocks dot com. Attachments and email works without editing.
       
  8. Chojin

    Chojin New Member

    Well, it does not work. I mean I cannot attach a file to a post, it does not appear on the attachment window, because as soon as I uploaded it, nothing happens.
    In javascript console I have this error:
    Code:
    Warning: Uncaught exception 'ErrorException' with message 'require_once(google/appengine/api/app_identity/AppIdentityService.php): failed to open stream: No such file or directory' in /base/data/home/apps/s~myapp/1.372745200003509748/library/bdGAE/Helper/SignedSession.php:40 Stack trace: #0 /base/data/home/apps/s~myapp/1.372745200003509748/library/bdGAE/Helper/SignedSession.php(40): XenForo_Application::handlePhpError(2, 'require_once(go...', '/base/data/home...', 40, Array) #1 /base/data/home/apps/s~myapp/1.372745200003509748/library/bdGAE/Helper/SignedSession.php(40): bdGAE_Helper_SignedSession::tryToRestoreSessionId() #2 /base/data/home/apps/s~myapp/1.372745200003509748/library/bdGAE/Listener.php(50): bdGAE_Helper_SignedSession::tryToRestoreSessionId() #3 [internal function]: bdGAE_Listener::init_dependencies(Object(XenForo_Dependencies_Public), Array) #4 /base/data/home/apps/s~myapp/1.372745200003509748/library/XenForo/CodeEvent.php(90): call_user_func_array(Array, Array) #5 /base/data/hom in /base/data/home/apps/s~myapp/1.372745200003509748/library/bdGAE/Helper/SignedSession.php on line 40
    Fatal error: bdGAE_Helper_SignedSession::tryToRestoreSessionId(): Failed opening required 'google/appengine/api/app_identity/AppIdentityService.php' (include_path='/base/data/home/apps/s~myapp/1.372745200003509748/library;.') in /base/data/home/apps/s~myapp/1.372745200003509748/library/bdGAE/Helper/SignedSession.php on line 40
    
    One important thing, I use XenForo 1.2.4 :D
     
    1. Hardcore

      Hardcore New Member

      #16 Hardcore, Nov 14, 2014
      Last edited by a moderator: Mar 12, 2016
      Hardcore @Chojin Add this above line 41 in bdGAE/Helper/Config.php:

      bdGAE_Helper_Patch::patchXenForoAutoloader();​

      This fixes the PHP include errors and attachment uploads, however, I am still unable to upload avatars.
       
  9. xfrocks

    xfrocks XenForo rocks!
    Staff Member

    Interesting, I run into similar error message on my site. Probably Google changed something with the PHP runtime and broke it. I will have to check and get back to you later ;)
     
  10. RoldanLT

    RoldanLT Active Member

    I'm using php 5.5.7 so it doesn't support?
     
    1. xfrocks

      xfrocks XenForo rocks!
      Staff Member

      xfrocks @RoldanLT What did you mean by "it"? This add-on? It works with PHP 5.5 I think. However, you don't have many choices for Google App Engine.
       
  11. RoldanLT

    RoldanLT Active Member

    Yes this addon.
    Is this forum already using this?
    Or do you have any demo forum that is already implementing this? :)
     
    1. xfrocks

      xfrocks XenForo rocks!
      Staff Member

      xfrocks @RoldanLT This add-on works with PHP 5.2+. No, this site does not run on GAE (too expensive, and slow!). My test site doesn't have a XenForo license so I can't share it, sorry :p
       
      RoldanLT likes this.
      1
Loading...