SslBump: Support bump-ssl-server-first and mimic SSL server certificates. Summary: These changes allow Squid working in SslBump mode to peek at the origin server certificate and mimic peeked server certificate properties in the generated fake certificate, all prior to establishing a secure connection with the client: http://wiki.squid-cache.org/Features/BumpSslServerFirst http://wiki.squid-cache.org/Features/MimicSslServerCert The changes are required to bump intercepted SSL connections without excessive browser warnings. The changes allow to disable bumping of some intercepted SSL connections, forcing Squid to go into a TCP tunnel mode for those connections. The changes also empower end user to examine and either honor or bypass most origin SSL server certificate errors. Prior to these changes, the responsibility for ignoring certificate validation errors belonged exclusively to Squid, necessarily leaving users in the dark if errors are ignored/bypassed. Squid can still be configured to emulate old bump-ssl-client-first behavior. However, a manual revision of ssl_bump options is required during upgrade because ssl_bump no longer supports an implicit "negate the last one" rule (and it is risky to let Squid guess what the admin true intent was or mix old- and new-style rules). Finally, fake certificate generation has been significantly improved. The new code guarantees that all identically configured Squids receiving identical origin server certificates will generate identical fake certificates, even if those Squid instances are running on different hosts, at different times, and do not communicate with each other. Such stable, reproducible certificates are required for distributed, scalable, or fail-safe Squid deployment. Overall, the changes are meant to make SslBump more powerful and safer. The code has been tested in several independent labs. Specific major changes are highlighted below: Make bumping algorithm selectable using ACLs. Even though bump-server-first is an overall better method, bumping the client first is useful for backward compatibility and possibly for serving internal Squid objects (such as icons inside Squid error pages). The following example bumps special and most other requests only, using the old bump-client-first approach for the special requests only: ssl_bump client-first specialOnes ssl_bump server-first mostOthers ssl_bump none all It allow use the old ssl_bump syntax: ssl_bump allow/deny acl ... but warns the user to update squid configuration. Added sslproxy_cert_adapt squid.conf option to overwrite default mimicking behavior when generating SSL certificates. See squid.conf.documented. iAdded sslproxy_cert_sign squid.conf option to control how generated SSL certificates are signed. See squid.conf.documented. Added ssl::certHasExpired, ssl::certNotYetValid, ssl::certDomainMismatch, ssl::certUntrusted, and ssl::certSelfSign predefined ACLs to squid.conf. Do not require http[s]_port's key option to be set if cert option is given. The fixed behavior for bumped connections now matches squid.conf docs. Generate stable fake certificates by using signing and true certificate hashes as the serial number and by using the configured CA private key for all fake certificates. Use minimal, trusted certificate for serving SSL errors to the client instead of trying to mimic the broken true certificate (which results in double error for the user: browser error dialog plus Squid error page). To mimic "untrusted" true certificates, generate an untrusted CA certificate from the configured trusted CA certificate. This both reduces configuration effort (compared to a configuration option) and results in identical untrusted fake certificates given identical Squid configurations. Intelligent handling of CONNECT denials: Do not connect to origin servers unless CONNECT is successfully authenticated. Delay errors.Added sslproxy_cert_sign squid.conf option to control how generated SSL certificates are signed. See squid.conf.documented. Added ssl::certHasExpired, ssl::certNotYetValid, ssl::certDomainMismatch, ssl::certUntrusted, and ssl::certSelfSign predefined ACLs to squid.conf. Do not require http[s]_port's key option to be set if cert option is given. The fixed behavior for bumped connections now matches squid.conf docs. Generate stable fake certificates by using signing and true certificate hashes as the serial number and by using the configured CA private key for all fake certificates. Use minimal, trusted certificate for serving SSL errors to the client instead of trying to mimic the broken true certificate (which results in double error for the user: browser error dialog plus Squid error page). To mimic "untrusted" true certificates, generate an untrusted CA certificate from the configured trusted CA certificate. This both reduces configuration effort (compared to a configuration option) and results in identical untrusted fake certificates given identical Squid configurations. Intelligent handling of CONNECT denials: Do not connect to origin servers unless CONNECT is successfully authenticated. Delay errors. Provide '%I' error page formatting code with enough information to avoid displaying '[unknown]' on SQUID_X509_V_ERR_DOMAIN_MISMATCH errors. Set logged status code (% class CbDataList { public: void *operator new (size_t); void operator delete (void *); CbDataList (C const &); ~CbDataList(); + /// If element is already in the list, returns false. + /// Otherwise, adds the element to the end of the list and returns true. + /// Exists to avoid double iteration of find() and push() combo. + bool push_back_unique(C const &element); bool find(C const &)const; bool findAndTune(C const &); + /// Iterates the entire list to return the last element holder. + CbDataList *tail(); CbDataList *next; C element; bool empty() const { return this == NULL; } private: CBDATA_CLASS(CbDataList); }; /// \ingroup POD template class CbDataListContainer { public: CbDataListContainer(); ~CbDataListContainer(); CbDataList *push_back (C const &); C pop_front(); bool empty() const; @@ -110,40 +116,64 @@ template void CbDataList::operator delete (void *address) { cbdataFree(address); } template CbDataList::CbDataList(C const &value) : next(NULL), element (value) {} template CbDataList::~CbDataList() { if (next) delete next; } template bool +CbDataList::push_back_unique(C const &toAdd) +{ + CbDataList *last; + for (last = this; last->next; last = last->next) { + if (last->element == toAdd) + return false; + } + + last->next = new CbDataList(toAdd); + return true; +} + +template +CbDataList * +CbDataList::tail() +{ + CbDataList *last; + for (last = this; last->next; last = last->next); + return last; +} + + +template +bool CbDataList::find (C const &toFind) const { CbDataList const *node = NULL; for (node = this; node; node = node->next) if (node->element == toFind) return true; return false; } template bool CbDataList::findAndTune(C const & toFind) { CbDataList *prev = NULL; for (CbDataList *node = this; node; node = node-> next) { if (node->element == toFind) { === modified file 'src/AccessLogEntry.cc' --- src/AccessLogEntry.cc 2012-01-20 18:55:04 +0000 +++ src/AccessLogEntry.cc 2012-07-01 15:31:06 +0000 @@ -1,19 +1,27 @@ #include "squid.h" #include "AccessLogEntry.h" #include "HttpRequest.h" +#include "ssl/support.h" + +#if USE_SSL +AccessLogEntry::Ssl::Ssl(): user(NULL), bumpMode(::Ssl::bumpEnd) +{ +} +#endif /* USE_SSL */ + void AccessLogEntry::getLogClientIp(char *buf, size_t bufsz) const { #if FOLLOW_X_FORWARDED_FOR if (Config.onoff.log_uses_indirect_client && request) request->indirect_client_addr.NtoA(buf, bufsz); else #endif if (tcpClient != NULL) tcpClient->remote.NtoA(buf, bufsz); else if (cache.caddr.IsNoAddr()) // e.g., ICAP OPTIONS lack client strncpy(buf, "-", 1); else cache.caddr.NtoA(buf, bufsz); } === modified file 'src/AccessLogEntry.h' --- src/AccessLogEntry.h 2012-04-25 05:29:20 +0000 +++ src/AccessLogEntry.h 2012-07-01 15:31:06 +0000 @@ -94,40 +94,51 @@ class IcpDetails { public: IcpDetails() : opcode(ICP_INVALID) {} icp_opcode opcode; } icp; /** \brief This subclass holds log info for HTCP protocol * \todo Inner class declarations should be moved outside */ class HtcpDetails { public: HtcpDetails() : opcode(NULL) {}; const char *opcode; } htcp; +#if USE_SSL + /// logging information specific to the SSL protocol + class Ssl { + public: + Ssl(); + + const char *user; ///< emailAddress from the SSL client certificate + int bumpMode; ///< whether and how the request was SslBumped + } ssl; +#endif + /** \brief This subclass holds log info for Squid internal stats * \todo Inner class declarations should be moved outside * \todo some details relevant to particular protocols need shuffling to other sub-classes * \todo this object field need renaming to 'squid' or something. */ class CacheDetails { public: CacheDetails() : caddr(), requestSize(0), replySize(0), requestHeadersSize(0), replyHeadersSize(0), highOffset(0), objectSize(0), code (LOG_TAG_NONE), msec(0), rfc931 (NULL), authuser (NULL), === modified file 'src/AclRegs.cc' --- src/AclRegs.cc 2012-01-20 18:55:04 +0000 +++ src/AclRegs.cc 2012-02-24 10:13:56 +0000 @@ -120,41 +120,41 @@ ACL::Prototype ACLRequestMIMEType::RegistryProtoype(&ACLRequestMIMEType::RegistryEntry_, "req_mime_type"); ACLStrategised ACLRequestMIMEType::RegistryEntry_(new ACLRegexData, ACLRequestHeaderStrategy::Instance(), "req_mime_type"); ACL::Prototype ACLSourceDomain::LiteralRegistryProtoype(&ACLSourceDomain::LiteralRegistryEntry_, "srcdomain"); ACLStrategised ACLSourceDomain::LiteralRegistryEntry_(new ACLDomainData, ACLSourceDomainStrategy::Instance(), "srcdomain"); ACL::Prototype ACLSourceDomain::RegexRegistryProtoype(&ACLSourceDomain::RegexRegistryEntry_, "srcdom_regex"); ACLStrategised ACLSourceDomain::RegexRegistryEntry_(new ACLRegexData,ACLSourceDomainStrategy::Instance() ,"srcdom_regex"); ACL::Prototype ACLSourceIP::RegistryProtoype(&ACLSourceIP::RegistryEntry_, "src"); ACLSourceIP ACLSourceIP::RegistryEntry_; ACL::Prototype ACLTime::RegistryProtoype(&ACLTime::RegistryEntry_, "time"); ACLStrategised ACLTime::RegistryEntry_(new ACLTimeData, ACLTimeStrategy::Instance(), "time"); ACL::Prototype ACLUrl::RegistryProtoype(&ACLUrl::RegistryEntry_, "url_regex"); ACLStrategised ACLUrl::RegistryEntry_(new ACLRegexData, ACLUrlStrategy::Instance(), "url_regex"); ACL::Prototype ACLUrlPath::LegacyRegistryProtoype(&ACLUrlPath::RegistryEntry_, "pattern"); ACL::Prototype ACLUrlPath::RegistryProtoype(&ACLUrlPath::RegistryEntry_, "urlpath_regex"); ACLStrategised ACLUrlPath::RegistryEntry_(new ACLRegexData, ACLUrlPathStrategy::Instance(), "urlpath_regex"); ACL::Prototype ACLUrlPort::RegistryProtoype(&ACLUrlPort::RegistryEntry_, "port"); ACLStrategised ACLUrlPort::RegistryEntry_(new ACLIntRange, ACLUrlPortStrategy::Instance(), "port"); #if USE_SSL ACL::Prototype ACLSslError::RegistryProtoype(&ACLSslError::RegistryEntry_, "ssl_error"); -ACLStrategised ACLSslError::RegistryEntry_(new ACLSslErrorData, ACLSslErrorStrategy::Instance(), "ssl_error"); +ACLStrategised ACLSslError::RegistryEntry_(new ACLSslErrorData, ACLSslErrorStrategy::Instance(), "ssl_error"); ACL::Prototype ACLCertificate::UserRegistryProtoype(&ACLCertificate::UserRegistryEntry_, "user_cert"); ACLStrategised ACLCertificate::UserRegistryEntry_(new ACLCertificateData (sslGetUserAttribute), ACLCertificateStrategy::Instance(), "user_cert"); ACL::Prototype ACLCertificate::CARegistryProtoype(&ACLCertificate::CARegistryEntry_, "ca_cert"); ACLStrategised ACLCertificate::CARegistryEntry_(new ACLCertificateData (sslGetCAAttribute), ACLCertificateStrategy::Instance(), "ca_cert"); #endif #if USE_SQUID_EUI ACL::Prototype ACLARP::RegistryProtoype(&ACLARP::RegistryEntry_, "arp"); ACLARP ACLARP::RegistryEntry_("arp"); ACL::Prototype ACLEui64::RegistryProtoype(&ACLEui64::RegistryEntry_, "eui64"); ACLEui64 ACLEui64::RegistryEntry_("eui64"); #endif #if USE_IDENT ACL::Prototype ACLIdent::UserRegistryProtoype(&ACLIdent::UserRegistryEntry_, "ident"); ACLIdent ACLIdent::UserRegistryEntry_(new ACLUserData, "ident"); ACL::Prototype ACLIdent::RegexRegistryProtoype(&ACLIdent::RegexRegistryEntry_, "ident_regex" ); ACLIdent ACLIdent::RegexRegistryEntry_(new ACLRegexData, "ident_regex"); #endif === modified file 'src/ClientRequestContext.h' --- src/ClientRequestContext.h 2012-05-08 01:13:51 +0000 +++ src/ClientRequestContext.h 2012-06-20 14:27:31 +0000 @@ -30,47 +30,49 @@ void hostHeaderVerifyFailed(const char *A, const char *B); void clientAccessCheck(); void clientAccessCheck2(); void clientAccessCheckDone(const allow_t &answer); void clientRedirectStart(); void clientRedirectDone(char *result); void checkNoCache(); void checkNoCacheDone(const allow_t &answer); #if USE_ADAPTATION void adaptationAccessCheck(); #endif #if USE_SSL /** * Initiates and start the acl checklist to check if the a CONNECT * request must be bumped. \retval true if the acl check scheduled, false if no ssl-bump required */ bool sslBumpAccessCheck(); /// The callback function for ssl-bump access check list - void sslBumpAccessCheckDone(bool doSslBump); + void sslBumpAccessCheckDone(const allow_t &answer); #endif ClientHttpRequest *http; ACLChecklist *acl_checklist; /* need ptr back so we can unreg if needed */ int redirect_state; bool host_header_verify_done; bool http_access_done; bool adapted_http_access_done; #if USE_ADAPTATION bool adaptation_acl_check_done; #endif bool redirect_done; bool no_cache_done; bool interpreted_req_hdrs; bool tosToClientDone; bool nfmarkToClientDone; #if USE_SSL bool sslBumpCheckDone; #endif + ErrorState *error; ///< saved error page for centralized/delayed processing + bool readNextRequest; ///< whether Squid should read after error handling private: CBDATA_CLASS(ClientRequestContext); }; #endif /* SQUID_CLIENTREQUESTCONTEXT_H */ === modified file 'src/HttpRequest.cc' --- src/HttpRequest.cc 2012-02-03 04:07:36 +0000 +++ src/HttpRequest.cc 2012-04-20 09:05:09 +0000 @@ -511,40 +511,48 @@ bool HttpRequest::bodyNibbled() const { return body_pipe != NULL && body_pipe->consumedSize() > 0; } void HttpRequest::detailError(err_type aType, int aDetail) { if (errType || errDetail) debugs(11, 5, HERE << "old error details: " << errType << '/' << errDetail); debugs(11, 5, HERE << "current error details: " << aType << '/' << aDetail); // checking type and detail separately may cause inconsistency, but // may result in more details available if they only become available later if (!errType) errType = aType; if (!errDetail) errDetail = aDetail; } +void +HttpRequest::clearError() +{ + debugs(11, 7, HERE << "old error details: " << errType << '/' << errDetail); + errType = ERR_NONE; + errDetail = ERR_DETAIL_NONE; +} + const char *HttpRequest::packableURI(bool full_uri) const { if (full_uri) return urlCanonical((HttpRequest*)this); if (urlpath.size()) return urlpath.termedBuf(); return "/"; } void HttpRequest::packFirstLineInto(Packer * p, bool full_uri) const { // form HTTP request-line packerPrintf(p, "%s %s HTTP/%d.%d\r\n", RequestMethodStr(method), packableURI(full_uri), http_ver.major, http_ver.minor); } === modified file 'src/HttpRequest.h' --- src/HttpRequest.h 2011-12-30 01:24:57 +0000 +++ src/HttpRequest.h 2012-04-20 09:02:25 +0000 @@ -83,61 +83,64 @@ bool conditional() const; ///< has at least one recognized If-* header /// whether the client is likely to be able to handle a 1xx reply bool canHandle1xx() const; /* Now that we care what host contains it is better off being protected. */ /* HACK: These two methods are only inline to get around Makefile dependancies */ /* caused by HttpRequest being used in places it really shouldn't. */ /* ideally they would be methods of URL instead. */ inline void SetHost(const char *src) { host_addr.SetEmpty(); host_addr = src; if ( host_addr.IsAnyAddr() ) { xstrncpy(host, src, SQUIDHOSTNAMELEN); host_is_numeric = 0; } else { host_addr.ToHostname(host, SQUIDHOSTNAMELEN); debugs(23, 3, "HttpRequest::SetHost() given IP: " << host_addr); host_is_numeric = 1; } + safe_free(canonical); // force its re-build }; inline const char* GetHost(void) const { return host; }; inline int GetHostIsNumeric(void) const { return host_is_numeric; }; #if USE_ADAPTATION /// Returns possibly nil history, creating it if adapt. logging is enabled Adaptation::History::Pointer adaptLogHistory() const; /// Returns possibly nil history, creating it if requested Adaptation::History::Pointer adaptHistory(bool createIfNone = false) const; /// Makes their history ours, throwing on conflicts void adaptHistoryImport(const HttpRequest &them); #endif #if ICAP_CLIENT /// Returns possibly nil history, creating it if icap logging is enabled Adaptation::Icap::History::Pointer icapHistory() const; #endif void recordLookup(const DnsLookupDetails &detail); /// sets error detail if no earlier detail was available void detailError(err_type aType, int aDetail); + /// clear error details, useful for retries/repeats + void clearError(); protected: void clean(); void init(); public: HttpRequestMethod method; char login[MAX_LOGIN_SZ]; private: char host[SQUIDHOSTNAMELEN]; int host_is_numeric; #if USE_ADAPTATION mutable Adaptation::History::Pointer adaptHistory_; ///< per-HTTP transaction info #endif #if ICAP_CLIENT mutable Adaptation::Icap::History::Pointer icapHistory_; ///< per-HTTP transaction info === modified file 'src/Makefile.am' --- src/Makefile.am 2012-05-30 10:25:42 +0000 +++ src/Makefile.am 2012-06-20 14:27:31 +0000 @@ -935,40 +935,41 @@ sed \ -e "s%[@]DEFAULT_HTTP_PORT[@]%$(DEFAULT_HTTP_PORT)%g" \ -e "s%[@]DEFAULT_ICP_PORT[@]%$(DEFAULT_ICP_PORT)%g" \ -e "s%[@]DEFAULT_CACHE_EFFECTIVE_USER[@]%$(CACHE_EFFECTIVE_USER)%g" \ -e "s%[@]DEFAULT_MIME_TABLE[@]%$(DEFAULT_MIME_TABLE)%g" \ -e "s%[@]DEFAULT_DNSSERVER[@]%$(DEFAULT_DNSSERVER)%g" \ -e "s%[@]DEFAULT_SSL_CRTD[@]%$(DEFAULT_SSL_CRTD)%g" \ -e "s%[@]DEFAULT_UNLINKD[@]%$(DEFAULT_UNLINKD)%g" \ -e "s%[@]DEFAULT_PINGER[@]%$(DEFAULT_PINGER)%g" \ -e "s%[@]DEFAULT_DISKD[@]%$(DEFAULT_DISKD)%g" \ -e "s%[@]DEFAULT_LOGFILED[@]%$(DEFAULT_LOGFILED)%g;" \ -e "s%[@]DEFAULT_CACHE_LOG[@]%$(DEFAULT_CACHE_LOG)%g" \ -e "s%[@]DEFAULT_ACCESS_LOG[@]%$(DEFAULT_ACCESS_LOG)%g" \ -e "s%[@]DEFAULT_STORE_LOG[@]%$(DEFAULT_STORE_LOG)%g" \ -e "s%[@]DEFAULT_PID_FILE[@]%$(DEFAULT_PID_FILE)%g" \ -e "s%[@]DEFAULT_NETDB_FILE[@]%$(DEFAULT_NETDB_FILE)%g" \ -e "s%[@]DEFAULT_SWAP_DIR[@]%$(DEFAULT_SWAP_DIR)%g" \ -e "s%[@]DEFAULT_SSL_DB_DIR[@]%$(DEFAULT_SSL_DB_DIR)%g" \ -e "s%[@]DEFAULT_ICON_DIR[@]%$(DEFAULT_ICON_DIR)%g" \ -e "s%[@]DEFAULT_CONFIG_DIR[@]%$(DEFAULT_CONFIG_DIR)%g" \ + -e "s%[@]DEFAULT_ERROR_DIR[@]%$(DEFAULT_ERROR_DIR)%g" \ -e "s%[@]DEFAULT_PREFIX[@]%$(DEFAULT_PREFIX)%g" \ -e "s%[@]DEFAULT_HOSTS[@]%$(DEFAULT_HOSTS)%g" \ -e "s%[@]SQUID[@]%SQUID\ $(VERSION)%g" \ < $(srcdir)/cf.data.pre >$@ repl_modules.cc: repl_modules.sh Makefile $(SHELL) $(srcdir)/repl_modules.sh $(REPL_POLICIES) > repl_modules.cc include $(top_srcdir)/doc/manuals/Substitute.am squid.8: $(srcdir)/squid.8.in Makefile $(SUBSTITUTE) < $(srcdir)/squid.8.in > $@ man_MANS = squid.8 EXTRA_DIST += squid.8.in CLEANFILES += squid.8 install-data-local: install-sysconfDATA install-dataDATA @if test -f $(DESTDIR)$(DEFAULT_MIME_TABLE) ; then \ echo "$@ will not overwrite existing $(DESTDIR)$(DEFAULT_MIME_TABLE)" ; \ === modified file 'src/Server.cc' --- src/Server.cc 2012-06-18 23:08:56 +0000 +++ src/Server.cc 2012-06-20 14:27:31 +0000 @@ -811,114 +811,113 @@ completeForwarding(); } // common part of noteAdaptation*Aborted and noteBodyConsumerAborted methods void ServerStateData::handleAdaptationAborted(bool bypassable) { debugs(11,5, HERE << "handleAdaptationAborted; bypassable: " << bypassable << ", entry empty: " << entry->isEmpty()); if (abortOnBadEntry("entry went bad while ICAP aborted")) return; // TODO: bypass if possible if (entry->isEmpty()) { debugs(11,9, HERE << "creating ICAP error entry after ICAP failure"); ErrorState *err = new ErrorState(ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR, request); - err->xerrno = ERR_DETAIL_ICAP_RESPMOD_EARLY; + err->detailError(ERR_DETAIL_ICAP_RESPMOD_EARLY); fwd->fail(err); fwd->dontRetry(true); } else if (request) { // update logged info directly request->detailError(ERR_ICAP_FAILURE, ERR_DETAIL_ICAP_RESPMOD_LATE); } abortTransaction("ICAP failure"); } // adaptation service wants us to deny HTTP client access to this response void ServerStateData::handleAdaptationBlocked(const Adaptation::Answer &answer) { debugs(11,5, HERE << answer.ruleId); if (abortOnBadEntry("entry went bad while ICAP aborted")) return; if (!entry->isEmpty()) { // too late to block (should not really happen) if (request) request->detailError(ERR_ICAP_FAILURE, ERR_DETAIL_RESPMOD_BLOCK_LATE); abortTransaction("late adaptation block"); return; } debugs(11,7, HERE << "creating adaptation block response"); err_type page_id = aclGetDenyInfoPage(&Config.denyInfoList, answer.ruleId.termedBuf(), 1); if (page_id == ERR_NONE) page_id = ERR_ACCESS_DENIED; ErrorState *err = new ErrorState(page_id, HTTP_FORBIDDEN, request); - err->xerrno = ERR_DETAIL_RESPMOD_BLOCK_EARLY; + err->detailError(ERR_DETAIL_RESPMOD_BLOCK_EARLY); fwd->fail(err); fwd->dontRetry(true); abortTransaction("timely adaptation block"); } void ServerStateData::noteAdaptationAclCheckDone(Adaptation::ServiceGroupPointer group) { adaptationAccessCheckPending = false; if (abortOnBadEntry("entry went bad while waiting for ICAP ACL check")) return; // TODO: Should nonICAP and postICAP path check this on the server-side? // That check now only happens on client-side, in processReplyAccess(). if (virginReply()->expectedBodyTooLarge(*request)) { sendBodyIsTooLargeError(); return; } // TODO: Should we check receivedBodyTooLarge on the server-side as well? if (!group) { debugs(11,3, HERE << "no adapation needed"); setFinalReply(virginReply()); processReplyBody(); return; } startAdaptation(group, originalRequest()); processReplyBody(); } #endif void ServerStateData::sendBodyIsTooLargeError() { ErrorState *err = new ErrorState(ERR_TOO_BIG, HTTP_FORBIDDEN, request); - err->xerrno = errno; fwd->fail(err); fwd->dontRetry(true); abortTransaction("Virgin body too large."); } // TODO: when HttpStateData sends all errors to ICAP, // we should be able to move this at the end of setVirginReply(). void ServerStateData::adaptOrFinalizeReply() { #if USE_ADAPTATION // TODO: merge with client side and return void to hide the on/off logic? // The callback can be called with a NULL service if adaptation is off. adaptationAccessCheckPending = Adaptation::AccessCheck::Start( Adaptation::methodRespmod, Adaptation::pointPreCache, originalRequest(), virginReply(), this); debugs(11,5, HERE << "adaptationAccessCheckPending=" << adaptationAccessCheckPending); if (adaptationAccessCheckPending) return; #endif === modified file 'src/acl/Acl.h' --- src/acl/Acl.h 2012-05-08 01:21:10 +0000 +++ src/acl/Acl.h 2012-05-12 16:05:19 +0000 @@ -99,41 +99,66 @@ char const *typeString; private: static Vector * Registry; static void *Initialized; typedef Vector::iterator iterator; typedef Vector::const_iterator const_iterator; void registerMe(); }; }; /// \ingroup ACLAPI typedef enum { // Authorization ACL result states ACCESS_DENIED, ACCESS_ALLOWED, ACCESS_DUNNO, // Authentication ACL result states ACCESS_AUTH_REQUIRED, // Missing Credentials -} allow_t; +} aclMatchCode; + +/// \ingroup ACLAPI +/// ACL check answer; TODO: Rename to Acl::Answer +class allow_t { +public: + // not explicit: allow "aclMatchCode to allow_t" conversions (for now) + allow_t(const aclMatchCode aCode): code(aCode), kind(0) {} + + allow_t(): code(ACCESS_DUNNO), kind(0) {} + + bool operator ==(const aclMatchCode aCode) const { + return code == aCode; + } + + bool operator !=(const aclMatchCode aCode) const { + return !(*this == aCode); + } + + operator aclMatchCode() const { + return code; + } + + aclMatchCode code; ///< ACCESS_* code + int kind; ///< which custom access list verb matched +}; inline std::ostream & operator <<(std::ostream &o, const allow_t a) { switch (a) { case ACCESS_DENIED: o << "DENIED"; break; case ACCESS_ALLOWED: o << "ALLOWED"; break; case ACCESS_DUNNO: o << "DUNNO"; break; case ACCESS_AUTH_REQUIRED: o << "AUTH_REQUIRED"; break; } return o; } === modified file 'src/acl/Checklist.h' --- src/acl/Checklist.h 2012-06-17 01:19:07 +0000 +++ src/acl/Checklist.h 2012-07-01 15:31:06 +0000 @@ -164,43 +164,44 @@ bool asyncInProgress() const; void asyncInProgress(bool const); /// whether markFinished() was called bool finished() const; /// called when no more ACLs should be checked; sets the final answer and /// prints a debugging message explaining the reason for that answer void markFinished(const allow_t &newAnswer, const char *reason); const allow_t ¤tAnswer() const { return allow_; } void changeState(AsyncState *); AsyncState *asyncState() const; // XXX: ACLs that need request or reply have to use ACLFilledChecklist and // should do their own checks so that we do not have to povide these two // for ACL::checklistMatches to use virtual bool hasRequest() const = 0; virtual bool hasReply() const = 0; -protected: - virtual void checkCallback(allow_t answer); private: + /// Calls non-blocking check callback with the answer and destroys self. + void checkCallback(allow_t answer); + void checkAccessList(); void checkForAsync(); public: const acl_access *accessList; ACLCB *callback; void *callback_data; /** * Performs non-blocking check starting with the current rule. * Used by nonBlockingCheck() to initiate the checks and by * async operation callbacks to resume checks after the async * operation updates the current Squid state. See nonBlockingCheck() * for details on final result determination. */ void matchNonBlocking(); private: /* internal methods */ /// possible outcomes when trying to match a single ACL node in a list === modified file 'src/acl/FilledChecklist.cc' --- src/acl/FilledChecklist.cc 2012-01-20 18:55:04 +0000 +++ src/acl/FilledChecklist.cc 2012-07-01 15:31:06 +0000 @@ -1,119 +1,98 @@ #include "squid-old.h" #include "HttpRequest.h" #include "HttpReply.h" #include "client_side.h" #if USE_AUTH #include "auth/UserRequest.h" #include "auth/AclProxyAuth.h" #endif #include "acl/FilledChecklist.h" #include "comm/Connection.h" #include "comm/forward.h" CBDATA_CLASS_INIT(ACLFilledChecklist); -void -ACLFilledChecklist::checkCallback(allow_t answer) -{ - debugs(28, 5, HERE << this << " answer=" << answer); - -#if USE_AUTH - /* During reconfigure, we can end up not finishing call - * sequences into the auth code */ - - if (auth_user_request != NULL) { - /* the filled_checklist lock */ - auth_user_request = NULL; - // It might have been connection based - // In the case of sslBump we need to preserve authentication info - // XXX: need to re-evaluate this. ACL tests should not be playing with - // XXX: wider scoped TCP connection state, even if the helper lookup is stuck. - if (conn() && !conn()->switchedToHttps()) { - conn()->auth_user_request = NULL; - } - } -#endif - - ACLChecklist::checkCallback(answer); // may delete us -} - void * ACLFilledChecklist::operator new (size_t size) { assert (size == sizeof(ACLFilledChecklist)); CBDATA_INIT_TYPE(ACLFilledChecklist); ACLFilledChecklist *result = cbdataAlloc(ACLFilledChecklist); return result; } void ACLFilledChecklist::operator delete (void *address) { ACLFilledChecklist *t = static_cast(address); cbdataFree(t); } ACLFilledChecklist::ACLFilledChecklist() : dst_peer(NULL), dst_rdns(NULL), request (NULL), reply (NULL), #if USE_AUTH auth_user_request (NULL), #endif #if SQUID_SNMP snmp_community(NULL), #endif #if USE_SSL - ssl_error(0), + sslErrors(NULL), #endif extacl_entry (NULL), conn_(NULL), fd_(-1), destinationDomainChecked_(false), sourceDomainChecked_(false) { my_addr.SetEmpty(); src_addr.SetEmpty(); dst_addr.SetEmpty(); rfc931[0] = '\0'; } ACLFilledChecklist::~ACLFilledChecklist() { assert (!asyncInProgress()); safe_free(dst_rdns); // created by xstrdup(). if (extacl_entry) cbdataReferenceDone(extacl_entry); HTTPMSGUNLOCK(request); HTTPMSGUNLOCK(reply); cbdataReferenceDone(conn_); +#if USE_SSL + cbdataReferenceDone(sslErrors); +#endif + debugs(28, 4, HERE << "ACLFilledChecklist destroyed " << this); } ConnStateData * ACLFilledChecklist::conn() const { return conn_; } void ACLFilledChecklist::conn(ConnStateData *aConn) { assert (conn() == NULL); conn_ = cbdataReference(aConn); } int ACLFilledChecklist::fd() const { @@ -162,41 +141,41 @@ * B) Using aclNBCheck() and callbacks: The caller allocates an * ACLFilledChecklist object (via operator new) and passes it to * aclNBCheck(). Control eventually passes to ACLChecklist::checkCallback(), * which will invoke the callback function as requested by the * original caller of aclNBCheck(). This callback function must * *not* delete the list. After the callback function returns, * checkCallback() will delete the list (i.e., self). */ ACLFilledChecklist::ACLFilledChecklist(const acl_access *A, HttpRequest *http_request, const char *ident): dst_peer(NULL), dst_rdns(NULL), request(NULL), reply(NULL), #if USE_AUTh auth_user_request(NULL), #endif #if SQUID_SNMP snmp_community(NULL), #endif #if USE_SSL - ssl_error(0), + sslErrors(NULL), #endif extacl_entry (NULL), conn_(NULL), fd_(-1), destinationDomainChecked_(false), sourceDomainChecked_(false) { my_addr.SetEmpty(); src_addr.SetEmpty(); dst_addr.SetEmpty(); rfc931[0] = '\0'; // cbdataReferenceDone() is in either fastCheck() or the destructor if (A) accessList = cbdataReference(A); if (http_request != NULL) { request = HTTPMSGLOCK(http_request); #if FOLLOW_X_FORWARDED_FOR if (Config.onoff.acl_uses_indirect_client) === modified file 'src/acl/FilledChecklist.h' --- src/acl/FilledChecklist.h 2011-12-30 01:24:57 +0000 +++ src/acl/FilledChecklist.h 2012-07-01 15:31:06 +0000 @@ -1,27 +1,30 @@ #ifndef SQUID_ACLFILLED_CHECKLIST_H #define SQUID_ACLFILLED_CHECKLIST_H #include "acl/Checklist.h" #if USE_AUTH #include "auth/UserRequest.h" #endif +#if USE_SSL +#include "ssl/support.h" +#endif class ExternalACLEntry; class ConnStateData; /** \ingroup ACLAPI ACLChecklist filled with specific data, representing Squid and transaction state for access checks along with some data-specific checking methods */ class ACLFilledChecklist: public ACLChecklist { public: void *operator new(size_t); void operator delete(void *); ACLFilledChecklist(); ACLFilledChecklist(const acl_access *, HttpRequest *, const char *ident); ~ACLFilledChecklist(); public: ConnStateData * conn() const; @@ -46,49 +49,47 @@ public: Ip::Address src_addr; Ip::Address dst_addr; Ip::Address my_addr; struct peer *dst_peer; char *dst_rdns; HttpRequest *request; HttpReply *reply; char rfc931[USER_IDENT_SZ]; #if USE_AUTH Auth::UserRequest::Pointer auth_user_request; #endif #if SQUID_SNMP char *snmp_community; #endif #if USE_SSL - int ssl_error; + /// SSL [certificate validation] errors, in undefined order + Ssl::Errors *sslErrors; #endif ExternalACLEntry *extacl_entry; private: - virtual void checkCallback(allow_t answer); - -private: CBDATA_CLASS(ACLFilledChecklist); ConnStateData * conn_; /**< hack for ident and NTLM */ int fd_; /**< may be available when conn_ is not */ bool destinationDomainChecked_; bool sourceDomainChecked_; private: /// not implemented; will cause link failures if used ACLFilledChecklist(const ACLFilledChecklist &); /// not implemented; will cause link failures if used ACLFilledChecklist &operator=(const ACLFilledChecklist &); }; /// convenience and safety wrapper for dynamic_cast inline ACLFilledChecklist *Filled(ACLChecklist *checklist) { // this should always be safe because ACLChecklist is an abstract class // and ACLFilledChecklist is its only [concrete] child === modified file 'src/acl/SslError.cc' --- src/acl/SslError.cc 2012-01-20 18:55:04 +0000 +++ src/acl/SslError.cc 2012-06-20 14:27:31 +0000 @@ -1,23 +1,23 @@ /* * $Id$ */ #include "squid-old.h" #include "acl/SslError.h" #include "acl/SslErrorData.h" #include "acl/Checklist.h" int ACLSslErrorStrategy::match (ACLData * &data, ACLFilledChecklist *checklist) { - return data->match (checklist->ssl_error); + return data->match (checklist->sslErrors); } ACLSslErrorStrategy * ACLSslErrorStrategy::Instance() { return &Instance_; } ACLSslErrorStrategy ACLSslErrorStrategy::Instance_; === modified file 'src/acl/SslError.h' --- src/acl/SslError.h 2009-03-08 21:53:27 +0000 +++ src/acl/SslError.h 2012-01-23 23:01:36 +0000 @@ -1,37 +1,38 @@ /* * $Id$ */ #ifndef SQUID_ACLSSL_ERROR_H #define SQUID_ACLSSL_ERROR_H #include "acl/Strategy.h" #include "acl/Strategised.h" +#include "ssl/support.h" -class ACLSslErrorStrategy : public ACLStrategy +class ACLSslErrorStrategy : public ACLStrategy { public: virtual int match (ACLData * &, ACLFilledChecklist *); static ACLSslErrorStrategy *Instance(); /* Not implemented to prevent copies of the instance. */ /* Not private to prevent brain dead g+++ warnings about * private constructors with no friends */ ACLSslErrorStrategy(ACLSslErrorStrategy const &); private: static ACLSslErrorStrategy Instance_; ACLSslErrorStrategy() {} ACLSslErrorStrategy&operator=(ACLSslErrorStrategy const &); }; class ACLSslError { private: static ACL::Prototype RegistryProtoype; - static ACLStrategised RegistryEntry_; + static ACLStrategised RegistryEntry_; }; #endif /* SQUID_ACLSSL_ERROR_H */ === modified file 'src/acl/SslErrorData.cc' --- src/acl/SslErrorData.cc 2012-01-20 18:55:04 +0000 +++ src/acl/SslErrorData.cc 2012-06-20 14:27:31 +0000 @@ -5,72 +5,76 @@ #include "squid-old.h" #include "acl/SslErrorData.h" #include "acl/Checklist.h" #include "wordlist.h" ACLSslErrorData::ACLSslErrorData() : values (NULL) {} ACLSslErrorData::ACLSslErrorData(ACLSslErrorData const &old) : values (NULL) { assert (!old.values); } ACLSslErrorData::~ACLSslErrorData() { if (values) delete values; } bool -ACLSslErrorData::match(Ssl::ssl_error_t toFind) +ACLSslErrorData::match(const Ssl::Errors *toFind) { - return values->findAndTune (toFind); + for (const Ssl::Errors *err = toFind; err; err = err->next ) { + if (values->findAndTune(err->element)) + return true; + } + return false; } /* explicit instantiation required for some systems */ /** \cond AUTODOCS-IGNORE */ // AYJ: 2009-05-20 : Removing. clashes with template instantiation for other ACLs. -// template cbdata_type CbDataList::CBDATA_CbDataList; +// template cbdata_type Ssl::Errors::CBDATA_CbDataList; /** \endcond */ wordlist * ACLSslErrorData::dump() { wordlist *W = NULL; - CbDataList *data = values; + Ssl::Errors *data = values; while (data != NULL) { wordlistAdd(&W, Ssl::GetErrorName(data->element)); data = data->next; } return W; } void ACLSslErrorData::parse() { - CbDataList **Tail; + Ssl::Errors **Tail; char *t = NULL; for (Tail = &values; *Tail; Tail = &((*Tail)->next)); while ((t = strtokFile())) { - CbDataList *q = new CbDataList(Ssl::ParseErrorString(t)); + Ssl::Errors *q = Ssl::ParseErrorString(t); *(Tail) = q; - Tail = &q->next; + Tail = &q->tail()->next; } } bool ACLSslErrorData::empty() const { return values == NULL; } -ACLData * +ACLSslErrorData * ACLSslErrorData::clone() const { /* Splay trees don't clone yet. */ assert (!values); return new ACLSslErrorData(*this); } === modified file 'src/acl/SslErrorData.h' --- src/acl/SslErrorData.h 2010-12-14 12:17:34 +0000 +++ src/acl/SslErrorData.h 2012-01-23 23:00:49 +0000 @@ -1,35 +1,36 @@ /* * $Id$ */ #ifndef SQUID_ACLSSL_ERRORDATA_H #define SQUID_ACLSSL_ERRORDATA_H #include "acl/Acl.h" #include "acl/Data.h" #include "CbDataList.h" #include "ssl/support.h" #include "ssl/ErrorDetail.h" +#include -class ACLSslErrorData : public ACLData +class ACLSslErrorData : public ACLData { public: MEMPROXY_CLASS(ACLSslErrorData); ACLSslErrorData(); ACLSslErrorData(ACLSslErrorData const &); ACLSslErrorData &operator= (ACLSslErrorData const &); virtual ~ACLSslErrorData(); - bool match(Ssl::ssl_error_t); + bool match(const Ssl::Errors *); wordlist *dump(); void parse(); bool empty() const; - virtual ACLData *clone() const; + virtual ACLSslErrorData *clone() const; - CbDataList *values; + Ssl::Errors *values; }; MEMPROXY_CLASS_INLINE(ACLSslErrorData); #endif /* SQUID_ACLSSL_ERRORDATA_H */ === modified file 'src/adaptation/Iterator.cc' --- src/adaptation/Iterator.cc 2012-01-20 18:55:04 +0000 +++ src/adaptation/Iterator.cc 2012-04-19 15:42:07 +0000 @@ -40,40 +40,46 @@ { Adaptation::Initiate::start(); thePlan = ServicePlan(theGroup, filter()); step(); } void Adaptation::Iterator::step() { ++iterations; debugs(93,5, HERE << '#' << iterations << " plan: " << thePlan); Must(!theLauncher); if (thePlan.exhausted()) { // nothing more to do sendAnswer(Answer::Forward(theMsg)); Must(done()); return; } + HttpRequest *request = dynamic_cast(theMsg); + if (!request) + request = theCause; + assert(request); + request->clearError(); + if (iterations > Adaptation::Config::service_iteration_limit) { debugs(93,DBG_CRITICAL, "Adaptation iterations limit (" << Adaptation::Config::service_iteration_limit << ") exceeded:\n" << "\tPossible service loop with " << theGroup->kind << " " << theGroup->id << ", plan=" << thePlan); throw TexcHere("too many adaptations"); } ServicePointer service = thePlan.current(); Must(service != NULL); debugs(93,5, HERE << "using adaptation service: " << service->cfg().key); theLauncher = initiateAdaptation( service->makeXactLauncher(theMsg, theCause)); Must(initiated(theLauncher)); Must(!done()); } void Adaptation::Iterator::noteAdaptationAnswer(const Answer &answer) === modified file 'src/adaptation/icap/Launcher.cc' --- src/adaptation/icap/Launcher.cc 2012-01-20 18:55:04 +0000 +++ src/adaptation/icap/Launcher.cc 2012-04-19 14:16:23 +0000 @@ -26,42 +26,44 @@ Adaptation::Icap::Launcher::~Launcher() { assert(!theXaction); } void Adaptation::Icap::Launcher::start() { Adaptation::Initiate::start(); Must(theInitiator.set()); launchXaction("first"); } void Adaptation::Icap::Launcher::launchXaction(const char *xkind) { Must(!theXaction); ++theLaunches; debugs(93,4, HERE << "launching " << xkind << " xaction #" << theLaunches); Adaptation::Icap::Xaction *x = createXaction(); x->attempts = theLaunches; - if (theLaunches > 1) + if (theLaunches > 1) { + x->clearError(); x->disableRetries(); + } if (theLaunches >= TheConfig.repeat_limit) x->disableRepeats("over icap_retry_limit"); theXaction = initiateAdaptation(x); Must(initiated(theXaction)); } void Adaptation::Icap::Launcher::noteAdaptationAnswer(const Answer &answer) { debugs(93,5, HERE << "launches: " << theLaunches << " answer: " << answer); // XXX: akError is unused by ICAPXaction in favor of noteXactAbort() Must(answer.kind != Answer::akError); sendAnswer(answer); clearAdaptation(theXaction); Must(done()); } void Adaptation::Icap::Launcher::noteInitiatorAborted() { === modified file 'src/adaptation/icap/ModXact.cc' --- src/adaptation/icap/ModXact.cc 2012-05-31 00:20:44 +0000 +++ src/adaptation/icap/ModXact.cc 2012-06-20 14:27:31 +0000 @@ -1906,40 +1906,51 @@ if (virgin.header == NULL) return false; virgin.header->firstLineBuf(mb); return true; } void Adaptation::Icap::ModXact::detailError(int errDetail) { HttpRequest *request = dynamic_cast(adapted.header); // if no adapted request, update virgin (and inherit its properties later) // TODO: make this and HttpRequest::detailError constant, like adaptHistory if (!request) request = const_cast(&virginRequest()); if (request) request->detailError(ERR_ICAP_FAILURE, errDetail); } +void Adaptation::Icap::ModXact::clearError() +{ + HttpRequest *request = dynamic_cast(adapted.header); + // if no adapted request, update virgin (and inherit its properties later) + if (!request) + request = const_cast(&virginRequest()); + + if (request) + request->clearError(); +} + /* Adaptation::Icap::ModXactLauncher */ Adaptation::Icap::ModXactLauncher::ModXactLauncher(HttpMsg *virginHeader, HttpRequest *virginCause, Adaptation::ServicePointer aService): AsyncJob("Adaptation::Icap::ModXactLauncher"), Adaptation::Icap::Launcher("Adaptation::Icap::ModXactLauncher", aService) { virgin.setHeader(virginHeader); virgin.setCause(virginCause); updateHistory(true); } Adaptation::Icap::Xaction *Adaptation::Icap::ModXactLauncher::createXaction() { Adaptation::Icap::ServiceRep::Pointer s = dynamic_cast(theService.getRaw()); Must(s != NULL); return new Adaptation::Icap::ModXact(virgin.header, virgin.cause, s); } void Adaptation::Icap::ModXactLauncher::swanSong() === modified file 'src/adaptation/icap/ModXact.h' --- src/adaptation/icap/ModXact.h 2011-05-13 10:38:28 +0000 +++ src/adaptation/icap/ModXact.h 2012-06-20 14:27:31 +0000 @@ -151,40 +151,42 @@ // comm handlers virtual void handleCommConnected(); virtual void handleCommWrote(size_t size); virtual void handleCommRead(size_t size); void handleCommWroteHeaders(); void handleCommWroteBody(); // service waiting void noteServiceReady(); void noteServiceAvailable(); public: InOut virgin; InOut adapted; // bypasses exceptions if needed and possible virtual void callException(const std::exception &e); /// record error detail in the virgin request if possible virtual void detailError(int errDetail); + // Icap::Xaction API + virtual void clearError(); private: virtual void start(); /// locates the request, either as a cause or as a virgin message itself const HttpRequest &virginRequest() const; // Must always be available void estimateVirginBody(); void makeAdaptedBodyPipe(const char *what); void waitForService(); // will not send anything [else] on the adapted pipe bool doneSending() const; void startWriting(); void writeMore(); void writePreviewBody(); void writePrimeBody(); void writeSomeBody(const char *label, size_t size); === modified file 'src/adaptation/icap/Xaction.h' --- src/adaptation/icap/Xaction.h 2011-07-12 19:02:48 +0000 +++ src/adaptation/icap/Xaction.h 2012-06-20 14:27:31 +0000 @@ -117,40 +117,42 @@ virtual bool doneReading() const; virtual bool doneWriting() const; bool doneWithIo() const; virtual bool doneAll() const; // called just before the 'done' transaction is deleted virtual void swanSong(); // returns a temporary string depicting transaction status, for debugging virtual const char *status() const; virtual void fillPendingStatus(MemBuf &buf) const; virtual void fillDoneStatus(MemBuf &buf) const; // useful for debugging virtual bool fillVirginHttpHeader(MemBuf&) const; public: // custom exception handling and end-of-call checks virtual void callException(const std::exception &e); virtual void callEnd(); + /// clear stored error details, if any; used for retries/repeats + virtual void clearError() {} void dnsLookupDone(const ipcache_addrs *ia); protected: // logging void setOutcome(const XactOutcome &xo); virtual void finalizeLogInfo(); public: ServiceRep &service(); private: void tellQueryAborted(); void maybeLog(); protected: Comm::ConnectionPointer connection; ///< ICAP server connection Adaptation::Icap::ServiceRep::Pointer theService; /* * We have two read buffers. We would prefer to read directly === modified file 'src/anyp/PortCfg.cc' --- src/anyp/PortCfg.cc 2012-04-25 05:29:20 +0000 +++ src/anyp/PortCfg.cc 2012-05-14 10:03:48 +0000 @@ -1,26 +1,29 @@ #include "squid.h" #include "anyp/PortCfg.h" #include "comm.h" #if HAVE_LIMITS #include #endif +#if USE_SSL +#include "ssl/support.h" +#endif CBDATA_NAMESPACED_CLASS_INIT(AnyP, PortCfg); int NHttpSockets = 0; int HttpSockets[MAXTCPLISTENPORTS]; AnyP::PortCfg::PortCfg(const char *aProtocol) #if USE_SSL : dynamicCertMemCacheSize(std::numeric_limits::max()) #endif { protocol = xstrdup(aProtocol); } AnyP::PortCfg::~PortCfg() { if (Comm::IsConnOpen(listenConn)) { listenConn->close(); listenConn = NULL; @@ -72,20 +75,59 @@ #if USE_SSL char *cert; char *key; int version; char *cipher; char *options; char *clientca; char *cafile; char *capath; char *crlfile; char *dhfile; char *sslflags; char *sslContextSessionId; SSL_CTX *sslContext; #endif #endif /*0*/ return b; } + +#if USE_SSL +void AnyP::PortCfg::configureSslServerContext() +{ + staticSslContext.reset( + sslCreateServerContext(cert, key, + version, cipher, options, sslflags, clientca, + cafile, capath, crlfile, dhfile, + sslContextSessionId)); + + if (!staticSslContext) { + char buf[128]; + fatalf("%s_port %s initialization error", protocol, s.ToURL(buf, sizeof(buf))); + } + + if (!sslBump) + return; + + if (cert) + Ssl::readCertChainAndPrivateKeyFromFiles(signingCert, signPkey, certsToChain, cert, key); + + if (!signingCert) { + char buf[128]; + fatalf("No valid signing SSL certificate configured for %s_port %s", protocol, s.ToURL(buf, sizeof(buf))); + } + + if (!signPkey) + debugs(3, DBG_IMPORTANT, "No SSL private key configured for " << protocol << "_port " << s); + + Ssl::generateUntrustedCert(untrustedSigningCert, untrustedSignPkey, + signingCert, signPkey); + + if (!untrustedSigningCert) { + char buf[128]; + fatalf("Unable to generate signing SSL certificate for untrusted sites for %s_port %s", protocol, s.ToURL(buf, sizeof(buf))); + } +} +#endif + === modified file 'src/anyp/PortCfg.h' --- src/anyp/PortCfg.h 2012-04-25 05:29:20 +0000 +++ src/anyp/PortCfg.h 2012-06-20 14:27:31 +0000 @@ -1,37 +1,41 @@ #ifndef SQUID_ANYP_PORTCFG_H #define SQUID_ANYP_PORTCFG_H #include "cbdata.h" #include "comm/Connection.h" #if USE_SSL #include "ssl/gadgets.h" #endif namespace AnyP { struct PortCfg { PortCfg(const char *aProtocol); ~PortCfg(); AnyP::PortCfg *clone() const; +#if USE_SSL + /// creates, configures, and validates SSL context and related port options + void configureSslServerContext(); +#endif PortCfg *next; Ip::Address s; char *protocol; /* protocol name */ char *name; /* visible name */ char *defaultsite; /* default web site */ unsigned int intercepted:1; /**< intercepting proxy port */ unsigned int spoof_client_ip:1; /**< spoof client ip if possible */ unsigned int accel:1; /**< HTTP accelerator */ unsigned int allow_direct:1; /**< Allow direct forwarding in accelerator mode */ unsigned int vhost:1; /**< uses host header */ unsigned int sslBump:1; /**< intercepts CONNECT requests */ unsigned int actAsOrigin:1; ///< update replies to conform with RFC 2616 unsigned int ignore_cc:1; /**< Ignore request Cache-Control directives */ int vport; /* virtual port support, -1 for dynamic, >0 static*/ bool connection_auth_disabled; /* Don't support connection oriented auth */ int disable_pmtu_discovery; @@ -53,35 +57,37 @@ #if USE_SSL char *cert; char *key; int version; char *cipher; char *options; char *clientca; char *cafile; char *capath; char *crlfile; char *dhfile; char *sslflags; char *sslContextSessionId; ///< "session id context" for staticSslContext bool generateHostCertificates; ///< dynamically make host cert for sslBump size_t dynamicCertMemCacheSize; ///< max size of generated certificates memory cache Ssl::SSL_CTX_Pointer staticSslContext; ///< for HTTPS accelerator or static sslBump Ssl::X509_Pointer signingCert; ///< x509 certificate for signing generated certificates Ssl::EVP_PKEY_Pointer signPkey; ///< private key for sighing generated certificates Ssl::X509_STACK_Pointer certsToChain; ///< x509 certificates to send with the generated cert + Ssl::X509_Pointer untrustedSigningCert; ///< x509 certificate for signing untrusted generated certificates + Ssl::EVP_PKEY_Pointer untrustedSignPkey; ///< private key for signing untrusted generated certificates #endif CBDATA_CLASS2(PortCfg); // namespaced }; } // namespace AnyP // Max number of TCP listening ports #define MAXTCPLISTENPORTS 128 // TODO: kill this global array. Need to check performance of array vs list though. extern int NHttpSockets; extern int HttpSockets[MAXTCPLISTENPORTS]; #endif /* SQUID_ANYP_PORTCFG_H */ === modified file 'src/cache_cf.cc' --- src/cache_cf.cc 2012-06-18 15:43:05 +0000 +++ src/cache_cf.cc 2012-07-16 14:52:13 +0000 @@ -38,40 +38,41 @@ #include "acl/Gadgets.h" #include "acl/MethodData.h" #if USE_ADAPTATION #include "adaptation/Config.h" #endif #if ICAP_CLIENT #include "adaptation/icap/Config.h" #endif #if USE_ECAP #include "adaptation/ecap/Config.h" #endif #include "anyp/PortCfg.h" #if USE_SSL #include "ssl/support.h" #include "ssl/Config.h" #endif #if USE_AUTH #include "auth/Config.h" #include "auth/Scheme.h" #endif +#include "base/RunnersRegistry.h" #include "ConfigParser.h" #include "CpuAffinityMap.h" #include "DiskIO/DiskIOModule.h" #include "eui/Config.h" #if USE_SQUID_ESI #include "esi/Parser.h" #endif #include "format/Format.h" #include "HttpRequestMethod.h" #include "ident/Config.h" #include "ip/Intercept.h" #include "ip/QosConfig.h" #include "ip/tools.h" #include "log/Config.h" #include "MemBuf.h" #include "mgr/Registration.h" #include "Parsing.h" #include "rfc1738.h" #if SQUID_SNMP #include "snmp.h" @@ -138,79 +139,92 @@ static const char *const B_KBYTES_STR = "KB"; static const char *const B_MBYTES_STR = "MB"; static const char *const B_GBYTES_STR = "GB"; static const char *const list_sep = ", \t\n\r"; static void parse_access_log(customlog ** customlog_definitions); static int check_null_access_log(customlog *customlog_definitions); static void dump_access_log(StoreEntry * entry, const char *name, customlog * definitions); static void free_access_log(customlog ** definitions); static void update_maxobjsize(void); static void configDoConfigure(void); static void parse_refreshpattern(refresh_t **); static uint64_t parseTimeUnits(const char *unit, bool allowMsec); static void parseTimeLine(time_msec_t * tptr, const char *units, bool allowMsec); static void parse_u_short(unsigned short * var); static void parse_string(char **); static void default_all(void); static void defaults_if_none(void); +static void defaults_postscriptum(void); static int parse_line(char *); static void parse_obsolete(const char *); static void parseBytesLine(size_t * bptr, const char *units); #if USE_SSL static void parseBytesOptionValue(size_t * bptr, const char *units, char const * value); #endif #if !USE_DNSHELPER static void parseBytesLineSigned(ssize_t * bptr, const char *units); #endif static size_t parseBytesUnits(const char *unit); static void free_all(void); void requirePathnameExists(const char *name, const char *path); static OBJH dump_config; #if USE_HTTP_VIOLATIONS static void dump_http_header_access(StoreEntry * entry, const char *name, header_mangler header[]); static void parse_http_header_access(header_mangler header[]); static void free_http_header_access(header_mangler header[]); static void dump_http_header_replace(StoreEntry * entry, const char *name, header_mangler header[]); static void parse_http_header_replace(header_mangler * header); static void free_http_header_replace(header_mangler * header); #endif static void parse_denyinfo(acl_deny_info_list ** var); static void dump_denyinfo(StoreEntry * entry, const char *name, acl_deny_info_list * var); static void free_denyinfo(acl_deny_info_list ** var); #if USE_WCCPv2 static void parse_IpAddress_list(Ip::Address_list **); static void dump_IpAddress_list(StoreEntry *, const char *, const Ip::Address_list *); static void free_IpAddress_list(Ip::Address_list **); #if CURRENTLY_UNUSED static int check_null_IpAddress_list(const Ip::Address_list *); #endif /* CURRENTLY_UNUSED */ #endif /* USE_WCCPv2 */ static void parsePortCfg(AnyP::PortCfg **, const char *protocol); #define parse_PortCfg(l) parsePortCfg((l), token) static void dump_PortCfg(StoreEntry *, const char *, const AnyP::PortCfg *); static void free_PortCfg(AnyP::PortCfg **); +#if USE_SSL +static void parse_sslproxy_cert_sign(sslproxy_cert_sign **cert_sign); +static void dump_sslproxy_cert_sign(StoreEntry *entry, const char *name, sslproxy_cert_sign *cert_sign); +static void free_sslproxy_cert_sign(sslproxy_cert_sign **cert_sign); +static void parse_sslproxy_cert_adapt(sslproxy_cert_adapt **cert_adapt); +static void dump_sslproxy_cert_adapt(StoreEntry *entry, const char *name, sslproxy_cert_adapt *cert_adapt); +static void free_sslproxy_cert_adapt(sslproxy_cert_adapt **cert_adapt); +static void parse_sslproxy_ssl_bump(acl_access **ssl_bump); +static void dump_sslproxy_ssl_bump(StoreEntry *entry, const char *name, acl_access *ssl_bump); +static void free_sslproxy_ssl_bump(acl_access **ssl_bump); +#endif /* USE_SSL */ + static void parse_b_size_t(size_t * var); static void parse_b_int64_t(int64_t * var); static bool parseNamedIntList(const char *data, const String &name, Vector &list); static void parse_CpuAffinityMap(CpuAffinityMap **const cpuAffinityMap); static void dump_CpuAffinityMap(StoreEntry *const entry, const char *const name, const CpuAffinityMap *const cpuAffinityMap); static void free_CpuAffinityMap(CpuAffinityMap **const cpuAffinityMap); static int parseOneConfigFile(const char *file_name, unsigned int depth); /* * LegacyParser is a parser for legacy code that uses the global * approach. This is static so that it is only exposed to cache_cf. * Other modules needing access to a ConfigParser should have it * provided to them in their parserFOO methods. */ static ConfigParser LegacyParser = ConfigParser(); void @@ -541,40 +555,42 @@ return err_count; } int parseConfigFile(const char *file_name) { int err_count = 0; debugs(5, 4, HERE); configFreeMemory(); ACLMethodData::ThePurgeCount = 0; default_all(); err_count = parseOneConfigFile(file_name, 0); defaults_if_none(); + defaults_postscriptum(); + /* * We must call configDoConfigure() before leave_suid() because * configDoConfigure() is where we turn username strings into * uid values. */ configDoConfigure(); if (!Config.chroot_dir) { leave_suid(); setUmask(Config.umask); _db_init(Debug::cache_log, Debug::debugOptions); enter_suid(); } if (opt_send_signal == -1) { Mgr::RegisterAction("config", "Current Squid Configuration", dump_config, 1, 1); } @@ -864,62 +880,50 @@ } Config2.effectiveGroupID = grp->gr_gid; } HttpRequestMethod::Configure(Config); #if USE_SSL debugs(3, 1, "Initializing https proxy context"); Config.ssl_client.sslContext = sslCreateClientContext(Config.ssl_client.cert, Config.ssl_client.key, Config.ssl_client.version, Config.ssl_client.cipher, Config.ssl_client.options, Config.ssl_client.flags, Config.ssl_client.cafile, Config.ssl_client.capath, Config.ssl_client.crlfile); for (peer *p = Config.peers; p != NULL; p = p->next) { if (p->use_ssl) { debugs(3, 1, "Initializing cache_peer " << p->name << " SSL context"); p->sslContext = sslCreateClientContext(p->sslcert, p->sslkey, p->sslversion, p->sslcipher, p->ssloptions, p->sslflags, p->sslcafile, p->sslcapath, p->sslcrlfile); } } for (AnyP::PortCfg *s = Config.Sockaddr.http; s != NULL; s = s->next) { - if (!s->cert && !s->key) + if (!s->sslBump) continue; debugs(3, 1, "Initializing http_port " << s->s << " SSL context"); - - s->staticSslContext.reset( - sslCreateServerContext(s->cert, s->key, - s->version, s->cipher, s->options, s->sslflags, s->clientca, - s->cafile, s->capath, s->crlfile, s->dhfile, - s->sslContextSessionId)); - - Ssl::readCertChainAndPrivateKeyFromFiles(s->signingCert, s->signPkey, s->certsToChain, s->cert, s->key); + s->configureSslServerContext(); } for (AnyP::PortCfg *s = Config.Sockaddr.https; s != NULL; s = s->next) { debugs(3, 1, "Initializing https_port " << s->s << " SSL context"); - - s->staticSslContext.reset( - sslCreateServerContext(s->cert, s->key, - s->version, s->cipher, s->options, s->sslflags, s->clientca, - s->cafile, s->capath, s->crlfile, s->dhfile, - s->sslContextSessionId)); + s->configureSslServerContext(); } #endif // prevent infinite fetch loops in the request parser // due to buffer full but not enough data recived to finish parse if (Config.maxRequestBufferSize <= Config.maxRequestHeaderSize) { fatalf("Client request buffer of %u bytes cannot hold a request with %u bytes of headers." \ " Change client_request_buffer_max or request_header_max_size limits.", (uint32_t)Config.maxRequestBufferSize, (uint32_t)Config.maxRequestHeaderSize); } #if USE_AUTH /* * disable client side request pipelining. There is a race with * Negotiate and NTLM when the client sends a second request on an * connection before the authenticate challenge is sent. With * pipelining OFF, the client may fail to authenticate, but squid's * state will be preserved. */ @@ -3786,40 +3790,55 @@ if (!protocol) { self_destruct(); return; } char *token = strtok(NULL, w_space); if (!token) { self_destruct(); return; } AnyP::PortCfg *s = new AnyP::PortCfg(protocol); parsePortSpecification(s, token); /* parse options ... */ while ((token = strtok(NULL, w_space))) { parse_port_option(s, token); } +#if USE_SSL + if (strcasecmp(protocol, "https") == 0) { + /* ssl-bump on https_port configuration requires either tproxy or intercept, and vice versa */ + const bool hijacked = s->spoof_client_ip || s->intercepted; + if (s->sslBump && !hijacked) { + debugs(3, DBG_CRITICAL, "FATAL: ssl-bump on https_port requires tproxy/intercept which is missing."); + self_destruct(); + } + if (hijacked && !s->sslBump) { + debugs(3, DBG_CRITICAL, "FATAL: tproxy/intercept on https_port requires ssl-bump which is missing."); + self_destruct(); + } + } +#endif + if (Ip::EnableIpv6&IPV6_SPECIAL_SPLITSTACK && s->s.IsAnyAddr()) { // clone the port options from *s to *(s->next) s->next = cbdataReference(s->clone()); s->next->s.SetIPv4(); debugs(3, 3, protocol << "_port: clone wildcard address for split-stack: " << s->s << " and " << s->next->s); } while (*head) head = &(*head)->next; *head = cbdataReference(s); } static void dump_generic_port(StoreEntry * e, const char *n, const AnyP::PortCfg * s) { char buf[MAX_IPSTRLEN]; storeAppendPrintf(e, "%s %s", n, @@ -4364,22 +4383,280 @@ } else if ((m = parseTimeUnits(token, false)) == 0) self_destruct(); cfg->oldest_service_failure = (m * d); } static void dump_icap_service_failure_limit(StoreEntry *entry, const char *name, const Adaptation::Icap::Config &cfg) { storeAppendPrintf(entry, "%s %d", name, cfg.service_failure_limit); if (cfg.oldest_service_failure > 0) { storeAppendPrintf(entry, " in %d seconds", (int)cfg.oldest_service_failure); } storeAppendPrintf(entry, "\n"); } static void free_icap_service_failure_limit(Adaptation::Icap::Config *cfg) { cfg->oldest_service_failure = 0; cfg->service_failure_limit = 0; } +#endif + +#if USE_SSL +static void parse_sslproxy_cert_adapt(sslproxy_cert_adapt **cert_adapt) +{ + char *al; + sslproxy_cert_adapt *ca = (sslproxy_cert_adapt *) xcalloc(1, sizeof(sslproxy_cert_adapt)); + if ((al = strtok(NULL, w_space)) == NULL) { + self_destruct(); + return; + } + + const char *param; + if ( char *s = strchr(al, '{')) { + *s = '\0'; // terminate the al string + s++; + param = s; + s = strchr(s, '}'); + if (!s) { + self_destruct(); + return; + } + *s = '\0'; + } + else + param = NULL; + + if (strcmp(al, Ssl::CertAdaptAlgorithmStr[Ssl::algSetValidAfter]) == 0) { + ca->alg = Ssl::algSetValidAfter; + ca->param = strdup("on"); + } + else if (strcmp(al, Ssl::CertAdaptAlgorithmStr[Ssl::algSetValidBefore]) == 0) { + ca->alg = Ssl::algSetValidBefore; + ca->param = strdup("on"); + } + else if (strcmp(al, Ssl::CertAdaptAlgorithmStr[Ssl::algSetCommonName]) == 0) { + ca->alg = Ssl::algSetCommonName; + if (param) { + if (strlen(param) > 64) { + debugs(3, DBG_CRITICAL, "FATAL: sslproxy_cert_adapt: setCommonName{" <param = strdup(param); + } + } else { + debugs(3, DBG_CRITICAL, "FATAL: sslproxy_cert_adapt: unknown cert adaptation algorithm: " << al); + self_destruct(); + return; + } + + aclParseAclList(LegacyParser, &ca->aclList); + + while(*cert_adapt) + cert_adapt = &(*cert_adapt)->next; + + *cert_adapt = ca; +} + +static void dump_sslproxy_cert_adapt(StoreEntry *entry, const char *name, sslproxy_cert_adapt *cert_adapt) +{ + for (sslproxy_cert_adapt *ca = cert_adapt; ca != NULL; ca = ca->next) { + storeAppendPrintf(entry, "%s ", name); + storeAppendPrintf(entry, "%s{%s} ", Ssl::sslCertAdaptAlgoritm(ca->alg), ca->param); + if (ca->aclList) + dump_acl_list(entry, ca->aclList); + storeAppendPrintf(entry, "\n"); + } +} + +static void free_sslproxy_cert_adapt(sslproxy_cert_adapt **cert_adapt) +{ + while(*cert_adapt) { + sslproxy_cert_adapt *ca = *cert_adapt; + *cert_adapt = ca->next; + safe_free(ca->param); + + if (ca->aclList) + aclDestroyAclList(&ca->aclList); + + safe_free(ca); + } +} + +static void parse_sslproxy_cert_sign(sslproxy_cert_sign **cert_sign) +{ + char *al; + sslproxy_cert_sign *cs = (sslproxy_cert_sign *) xcalloc(1, sizeof(sslproxy_cert_sign)); + if ((al = strtok(NULL, w_space)) == NULL) { + self_destruct(); + return; + } + + if (strcmp(al, Ssl::CertSignAlgorithmStr[Ssl::algSignTrusted]) == 0) + cs->alg = Ssl::algSignTrusted; + else if (strcmp(al, Ssl::CertSignAlgorithmStr[Ssl::algSignUntrusted]) == 0) + cs->alg = Ssl::algSignUntrusted; + else if (strcmp(al, Ssl::CertSignAlgorithmStr[Ssl::algSignSelf]) == 0) + cs->alg = Ssl::algSignSelf; + else { + debugs(3, DBG_CRITICAL, "FATAL: sslproxy_cert_sign: unknown cert signing algorithm: " << al); + self_destruct(); + return; + } + + aclParseAclList(LegacyParser, &cs->aclList); + + while(*cert_sign) + cert_sign = &(*cert_sign)->next; + + *cert_sign = cs; +} + +static void dump_sslproxy_cert_sign(StoreEntry *entry, const char *name, sslproxy_cert_sign *cert_sign) +{ + sslproxy_cert_sign *cs; + for (cs = cert_sign; cs != NULL; cs = cs->next) { + storeAppendPrintf(entry, "%s ", name); + storeAppendPrintf(entry, "%s ", Ssl::certSignAlgorithm(cs->alg)); + if (cs->aclList) + dump_acl_list(entry, cs->aclList); + storeAppendPrintf(entry, "\n"); + } +} + +static void free_sslproxy_cert_sign(sslproxy_cert_sign **cert_sign) +{ + while(*cert_sign) { + sslproxy_cert_sign *cs = *cert_sign; + *cert_sign = cs->next; + + if (cs->aclList) + aclDestroyAclList(&cs->aclList); + + safe_free(cs); + } +} + +class sslBumpCfgRr: public ::RegisteredRunner +{ +public: + static Ssl::BumpMode lastDeprecatedRule; + /* RegisteredRunner API */ + virtual void run(const RunnerRegistry &); +}; + +Ssl::BumpMode sslBumpCfgRr::lastDeprecatedRule = Ssl::bumpEnd; + +RunnerRegistrationEntry(rrFinalizeConfig, sslBumpCfgRr); + +void sslBumpCfgRr::run(const RunnerRegistry &r) +{ + if (lastDeprecatedRule != Ssl::bumpEnd) { + assert( lastDeprecatedRule == Ssl::bumpClientFirst || lastDeprecatedRule == Ssl::bumpNone); + static char buf[1024]; + if (lastDeprecatedRule == Ssl::bumpClientFirst) { + strcpy(buf, "ssl_bump deny all"); + debugs(3, DBG_CRITICAL, "WARNING: auto-converting deprecated implicit " + "\"ssl_bump deny all\" to \"ssl_bump none all\". New ssl_bump configurations " + "must not use implicit rules. Update your ssl_bump rules."); + } else { + strcpy(buf, "ssl_bump allow all"); + debugs(3, DBG_CRITICAL, "WARNING: auto-converting deprecated implicit " + "\"ssl_bump allow all\" to \"ssl_bump client-first all\" which is usually " + "inferior to the newer server-first bumping mode. New ssl_bump" + " configurations must not use implicit rules. Update your ssl_bump rules."); + } + parse_line(buf); + } +} + +static void parse_sslproxy_ssl_bump(acl_access **ssl_bump) +{ + typedef const char *BumpCfgStyle; + BumpCfgStyle bcsNone = NULL; + BumpCfgStyle bcsNew = "new client/server-first/none"; + BumpCfgStyle bcsOld = "deprecated allow/deny"; + static BumpCfgStyle bumpCfgStyleLast = bcsNone; + BumpCfgStyle bumpCfgStyleNow = bcsNone; + char *bm; + if ((bm = strtok(NULL, w_space)) == NULL) { + self_destruct(); + return; + } + + // if this is the first rule proccessed + if (*ssl_bump == NULL) { + bumpCfgStyleLast = bcsNone; + sslBumpCfgRr::lastDeprecatedRule = Ssl::bumpEnd; + } + + acl_access *A = new acl_access; + A->allow = allow_t(ACCESS_ALLOWED); + + if (strcmp(bm, Ssl::BumpModeStr[Ssl::bumpClientFirst]) == 0) { + A->allow.kind = Ssl::bumpClientFirst; + bumpCfgStyleNow = bcsNew; + } else if (strcmp(bm, Ssl::BumpModeStr[Ssl::bumpServerFirst]) == 0) { + A->allow.kind = Ssl::bumpServerFirst; + bumpCfgStyleNow = bcsNew; + } else if (strcmp(bm, Ssl::BumpModeStr[Ssl::bumpNone]) == 0) { + A->allow.kind = Ssl::bumpNone; + bumpCfgStyleNow = bcsNew; + } else if (strcmp(bm, "allow") == 0) { + if (sslBumpCfgRr::lastDeprecatedRule == Ssl::bumpEnd) + debugs(3, DBG_CRITICAL, "WARNING: auto-converting deprecated " + "\"ssl_bump allow \" to \"ssl_bump client-first \" which " + "is usually inferior to the newer server-first " + "bumping mode. Update your ssl_bump rules."); + A->allow.kind = Ssl::bumpClientFirst; + bumpCfgStyleNow = bcsOld; + sslBumpCfgRr::lastDeprecatedRule = Ssl::bumpClientFirst; + } else if (strcmp(bm, "deny") == 0) { + if (sslBumpCfgRr::lastDeprecatedRule == Ssl::bumpEnd) + debugs(3, DBG_CRITICAL, "WARNING: auto-converting deprecated " + "\"ssl_bump deny \" to \"ssl_bump none \". Update " + "your ssl_bump rules."); + A->allow.kind = Ssl::bumpNone; + bumpCfgStyleNow = bcsOld; + sslBumpCfgRr::lastDeprecatedRule = Ssl::bumpNone; + } else { + debugs(3, DBG_CRITICAL, "FATAL: unknown ssl_bump mode: " << bm); + self_destruct(); + return; + } + + if (bumpCfgStyleLast != bcsNone && bumpCfgStyleNow != bumpCfgStyleLast) { + debugs(3, DBG_CRITICAL, "FATAL: do not mix " << bumpCfgStyleNow << " actions with " << + bumpCfgStyleLast << " actions. Update your ssl_bump rules."); + self_destruct(); + return; + } + + bumpCfgStyleLast = bumpCfgStyleNow; + + aclParseAclList(LegacyParser, &A->aclList); + + acl_access *B, **T; + for (B = *ssl_bump, T = ssl_bump; B; T = &B->next, B = B->next); + *T = A; +} + +static void dump_sslproxy_ssl_bump(StoreEntry *entry, const char *name, acl_access *ssl_bump) +{ + acl_access *sb; + for (sb = ssl_bump; sb != NULL; sb = sb->next) { + storeAppendPrintf(entry, "%s ", name); + storeAppendPrintf(entry, "%s ", Ssl::bumpMode(sb->allow.kind)); + if (sb->aclList) + dump_acl_list(entry, sb->aclList); + storeAppendPrintf(entry, "\n"); + } +} + +static void free_sslproxy_ssl_bump(acl_access **ssl_bump) +{ + free_acl_access(ssl_bump); +} #endif === modified file 'src/cf.data.depend' --- src/cf.data.depend 2012-04-25 05:29:20 +0000 +++ src/cf.data.depend 2012-06-20 14:27:31 +0000 @@ -51,20 +51,23 @@ peer peer_access cache_peer acl PortCfg QosConfig refreshpattern removalpolicy size_t IpAddress_list string string time_msec time_t tristate uri_whitespace u_short wccp2_method wccp2_amethod wccp2_service wccp2_service_info wordlist +sslproxy_ssl_bump acl +sslproxy_cert_sign acl +sslproxy_cert_adapt acl === modified file 'src/cf.data.pre' --- src/cf.data.pre 2012-05-06 01:29:22 +0000 +++ src/cf.data.pre 2012-07-12 13:35:52 +0000 @@ -630,40 +630,47 @@ tag= Apply a tag to a request (for both ERR and OK results) Only sets a tag, does not alter existing tags. log= String to be logged in access.log. Available as %ea in logformat specifications If protocol=3.0 (the default) then URL escaping is used to protect each value in both requests and responses. If using protocol=2.5 then all values need to be enclosed in quotes if they may contain whitespace, or the whitespace escaped using \. And quotes or \ characters within the keyword value must be \ escaped. When using the concurrency= option the protocol is changed by introducing a query channel tag infront of the request/response. The query channel tag is a number between 0 and concurrency-1. DOC_END NAME: acl TYPE: acl LOC: Config.aclList +IF USE_SSL +DEFAULT: ssl::certHasExpired ssl_error X509_V_ERR_CERT_HAS_EXPIRED +DEFAULT: ssl::certNotYetValid ssl_error X509_V_ERR_CERT_NOT_YET_VALID +DEFAULT: ssl::certDomainMismatch ssl_error SQUID_X509_V_ERR_DOMAIN_MISMATCH +DEFAULT: ssl::certUntrusted ssl_error X509_V_ERR_INVALID_CA X509_V_ERR_SELF_SIGNED_CERT_IN_CHAIN X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY X509_V_ERR_CERT_UNTRUSTED +DEFAULT: ssl::certSelfSigned ssl_error X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT +ENDIF DEFAULT: all src all DEFAULT: manager url_regex -i ^cache_object:// +i ^https?://[^/]+/squid-internal-mgr/ DEFAULT: localhost src 127.0.0.1/32 ::1 DEFAULT: to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1 DEFAULT_DOC: ACLs all, manager, localhost, and to_localhost are predefined. DOC_START Defining an Access List Every access list definition must begin with an aclname and acltype, followed by either type-specific arguments or a quoted filename that they are read from. acl aclname acltype argument ... acl aclname acltype "file" ... When using "file", the file should contain one item per line. By default, regular expressions are CASE-SENSITIVE. To make them case-insensitive, use the -i option. To return case-sensitive use the +i option between patterns, or make a new ACL line without -i. @@ -854,40 +861,63 @@ acl aclname ca_cert attribute values... # match against attributes a users issuing CA SSL certificate # attribute is one of DN/C/O/CN/L/ST [fast] acl aclname ext_user username ... acl aclname ext_user_regex [-i] pattern ... # string match on username returned by external acl helper [slow] # use REQUIRED to accept any non-null user name. acl aclname tag tagvalue ... # string match on tag returned by external acl helper [slow] acl aclname hier_code codename ... # string match against squid hierarchy code(s); [fast] # e.g., DIRECT, PARENT_HIT, NONE, etc. # # NOTE: This has no effect in http_access rules. It only has # effect in rules that affect the reply data stream such as # http_reply_access. +IF USE_SSL + acl aclname ssl_error errorname + # match against SSL certificate validation error [fast] + # + # For valid error names see in @DEFAULT_ERROR_DIR@/templates/error-details.txt + # template file. + # + # The following can be used as shortcuts for certificate properties: + # [ssl::]certHasExpired: the "not after" field is in the past + # [ssl::]certNotYetValid: the "not before" field is in the future + # [ssl::]certUntrusted: The certificate issuer is not to be trusted. + # [ssl::]certSelfSigned: The certificate is self signed. + # [ssl::]certDomainMismatch: The certificate CN domain does not + # match the name the name of the host we are connecting to. + # + # The ssl::certHasExpired, ssl::certNotYetValid, ssl::certDomainMismatch, + # ssl::certUntrusted, and ssl::certSelfSigned can also be used as + # predefined ACLs, just like the 'all' ACL. + # + # NOTE: The ssl_error ACL is only supported with sslproxy_cert_error, + # sslproxy_cert_sign, and sslproxy_cert_adapt options. +ENDIF + Examples: acl macaddress arp 09:00:2b:23:45:67 acl myexample dst_as 1241 acl password proxy_auth REQUIRED acl fileupload req_mime_type -i ^multipart/form-data$ acl javascript rep_mime_type -i ^application/x-javascript$ NOCOMMENT_START # # Recommended minimum configuration: # # Example rule allowing access from your local networks. # Adapt to list your (internal) IP networks from where browsing # should be allowed acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines @@ -1299,48 +1329,48 @@ probably want to listen on port 80 also, or instead. The -a command line option may be used to specify additional port(s) where Squid listens for proxy request. Such ports will be plain proxy ports with no options. You may specify multiple socket addresses on multiple lines. Modes: intercept Support for IP-Layer interception of outgoing requests without browser settings. NP: disables authentication and IPv6 on the port. tproxy Support Linux TPROXY for spoofing outgoing connections using the client IP address. NP: disables authentication and maybe IPv6 on the port. accel Accelerator / reverse proxy mode - ssl-bump Intercept each CONNECT request matching ssl_bump ACL, + ssl-bump For each CONNECT request allowed by ssl_bump ACLs, establish secure connection with the client and with - the server, decrypt HTTP messages as they pass through + the server, decrypt HTTPS messages as they pass through Squid, and treat them as unencrypted HTTP messages, becoming the man-in-the-middle. The ssl_bump option is required to fully enable - the SslBump feature. + bumping of CONNECT requests. Omitting the mode flag causes default forward proxy mode to be used. Accelerator Mode Options: defaultsite=domainname What to use for the Host: header if it is not present in a request. Determines what site (not origin server) accelerators should consider the default. no-vhost Disable using HTTP/1.1 Host header for virtual domain support. protocol= Protocol to reconstruct accelerated requests with. Defaults to http for http_port and https for https_port vport Virtual host port support. Using the http_port number instead of the port passed on Host: headers. @@ -1369,42 +1399,41 @@ SSL Bump Mode Options: In addition to these options ssl-bump requires TLS/SSL options. generate-host-certificates[=] Dynamically create SSL server certificates for the destination hosts of bumped CONNECT requests.When enabled, the cert and key options are used to sign generated certificates. Otherwise generated certificate will be selfsigned. If there is a CA certificate lifetime of the generated certificate equals lifetime of the CA certificate. If generated certificate is selfsigned lifetime is three years. This option is enabled by default when ssl-bump is used. See the ssl-bump option above for more information. dynamic_cert_mem_cache_size=SIZE Approximate total RAM size spent on cached generated certificates. If set to zero, caching is disabled. The - default value is 4MB. An average XXX-bit certificate - consumes about XXX bytes of RAM. + default value is 4MB. TLS / SSL Options: cert= Path to SSL certificate (PEM format). key= Path to SSL private key file (PEM format) if not specified, the certificate file is assumed to be a combined certificate and key file. version= The version of SSL/TLS supported 1 automatic (default) 2 SSLv2 only 3 SSLv3 only 4 TLSv1.0 only 5 TLSv1.1 only 6 TLSv1.2 only cipher= Colon separated list of supported ciphers. NOTE: some ciphers such as EDH ciphers depend on @@ -1512,40 +1541,151 @@ NOCOMMENT_END DOC_END NAME: https_port IFDEF: USE_SSL TYPE: PortCfg DEFAULT: none LOC: Config.Sockaddr.https DOC_START Usage: [ip:]port cert=certificate.pem [key=key.pem] [mode] [options...] The socket address where Squid will listen for client requests made over TLS or SSL connections. Commonly referred to as HTTPS. This is most useful for situations where you are running squid in accelerator mode and you want to do the SSL work at the accelerator level. You may specify multiple socket addresses on multiple lines, each with their own SSL certificate and/or options. + Modes: + + accel Accelerator / reverse proxy mode + + intercept Support for IP-Layer interception of + outgoing requests without browser settings. + NP: disables authentication and IPv6 on the port. + + tproxy Support Linux TPROXY for spoofing outgoing + connections using the client IP address. + NP: disables authentication and maybe IPv6 on the port. + + ssl-bump For each intercepted connection allowed by ssl_bump + ACLs, establish a secure connection with the client and with + the server, decrypt HTTPS messages as they pass through + Squid, and treat them as unencrypted HTTP messages, + becoming the man-in-the-middle. + + An "ssl_bump server-first" match is required to + fully enable bumping of intercepted SSL connections. + + Requires tproxy or intercept. + + Omitting the mode flag causes default forward proxy mode to be used. + + + See http_port for a list of generic options + + + SSL Options: + + cert= Path to SSL certificate (PEM format). + + key= Path to SSL private key file (PEM format) + if not specified, the certificate file is + assumed to be a combined certificate and + key file. + + version= The version of SSL/TLS supported + 1 automatic (default) + 2 SSLv2 only + 3 SSLv3 only + 4 TLSv1 only + + cipher= Colon separated list of supported ciphers. + + options= Various SSL engine options. The most important + being: + NO_SSLv2 Disallow the use of SSLv2 + NO_SSLv3 Disallow the use of SSLv3 + NO_TLSv1 Disallow the use of TLSv1 + SINGLE_DH_USE Always create a new key when using + temporary/ephemeral DH key exchanges + See src/ssl_support.c or OpenSSL SSL_CTX_set_options + documentation for a complete list of options. + + clientca= File containing the list of CAs to use when + requesting a client certificate. + + cafile= File containing additional CA certificates to + use when verifying client certificates. If unset + clientca will be used. + + capath= Directory containing additional CA certificates + and CRL lists to use when verifying client certificates. + + crlfile= File of additional CRL lists to use when verifying + the client certificate, in addition to CRLs stored in + the capath. Implies VERIFY_CRL flag below. + + dhparams= File containing DH parameters for temporary/ephemeral + DH key exchanges. + + sslflags= Various flags modifying the use of SSL: + DELAYED_AUTH + Don't request client certificates + immediately, but wait until acl processing + requires a certificate (not yet implemented). + NO_DEFAULT_CA + Don't use the default CA lists built in + to OpenSSL. + NO_SESSION_REUSE + Don't allow for session reuse. Each connection + will result in a new SSL session. + VERIFY_CRL + Verify CRL lists when accepting client + certificates. + VERIFY_CRL_ALL + Verify CRL lists for all certificates in the + client certificate chain. + + sslcontext= SSL session ID context identifier. + + generate-host-certificates[=] + Dynamically create SSL server certificates for the + destination hosts of bumped SSL requests.When + enabled, the cert and key options are used to sign + generated certificates. Otherwise generated + certificate will be selfsigned. + If there is CA certificate life time of generated + certificate equals lifetime of CA certificate. If + generated certificate is selfsigned lifetime is three + years. + This option is enabled by default when SslBump is used. + See the sslBump option above for more information. + + dynamic_cert_mem_cache_size=SIZE + Approximate total RAM size spent on cached generated + certificates. If set to zero, caching is disabled. The + default value is 4MB. + See http_port for a list of available options. DOC_END NAME: tcp_outgoing_tos tcp_outgoing_ds tcp_outgoing_dscp TYPE: acl_tos DEFAULT: none LOC: Ip::Qos::TheConfig.tosToServer DOC_START Allows you to select a TOS/Diffserv value for packets outgoing on the server side, based on an ACL. tcp_outgoing_tos ds-field [!]aclname ... Example where normal_service_net uses the TOS value 0x00 and good_service_net uses 0x20 acl normal_service_net src 10.0.0.0/24 acl good_service_net src 10.0.1.0/24 tcp_outgoing_tos 0x00 normal_service_net tcp_outgoing_tos 0x20 good_service_net @@ -1927,66 +2067,94 @@ DEFAULT: none LOC: Config.ssl_client.cafile TYPE: string DOC_START file containing CA certificates to use when verifying server certificates while proxying https:// URLs DOC_END NAME: sslproxy_capath IFDEF: USE_SSL DEFAULT: none LOC: Config.ssl_client.capath TYPE: string DOC_START directory containing CA certificates to use when verifying server certificates while proxying https:// URLs DOC_END NAME: ssl_bump IFDEF: USE_SSL -TYPE: acl_access +TYPE: sslproxy_ssl_bump LOC: Config.accessList.ssl_bump DEFAULT: none DOC_START - This ACL controls which CONNECT requests to an http_port - marked with an sslBump flag are actually "bumped". Please - see the sslBump flag of an http_port option for more details - about decoding proxied SSL connections. - - By default, no requests are bumped. + This option is consulted when a CONNECT request is received on + an http_port (or a new connection is intercepted at an + https_port), provided that port was configured with an ssl-bump + flag. The subsequent data on the connection is either treated as + HTTPS and decrypted OR tunneled at TCP level without decryption, + depending on the first bumping "mode" which ACLs match. + + ssl_bump [!]acl ... + + The following bumping modes are supported: + + client-first + Allow bumping of the connection. Establish a secure connection + with the client first, then connect to the server. This old mode + does not allow Squid to mimic server SSL certificate and does + not work with intercepted SSL connections. + + server-first + Allow bumping of the connection. Establish a secure connection + with the server first, then establish a secure connection with + the client, using a mimicked server certificate. Works with both + CONNECT requests and intercepted SSL connections. + + none + Become a TCP tunnel without decoding the connection. + Works with both CONNECT requests and intercepted SSL + connections. This is the default behavior when no + ssl_bump option is given or no ssl_bump ACLs match. + + By default, no connections are bumped. + + The first matching ssl_bump option wins. If no ACLs match, the + connection is not bumped. Unlike most allow/deny ACL lists, ssl_bump + does not have an implicit "negate the last given option" rule. You + must make that rule explicit if you convert old ssl_bump allow/deny + rules that rely on such an implicit rule. - See also: http_port ssl-bump - This clause supports both fast and slow acl types. See http://wiki.squid-cache.org/SquidFaq/SquidAcl for details. + See also: http_port ssl-bump, https_port ssl-bump - # Example: Bump all requests except those originating from localhost and - # those going to webax.com or example.com sites. - acl localhost src 127.0.0.1/32 - acl broken_sites dstdomain .webax.com + # Example: Bump all requests except those originating from + # localhost and those going to example.com. + acl broken_sites dstdomain .example.com - ssl_bump deny localhost - ssl_bump deny broken_sites - ssl_bump allow all + ssl_bump none localhost + ssl_bump none broken_sites + ssl_bump server-first all DOC_END NAME: sslproxy_flags IFDEF: USE_SSL DEFAULT: none LOC: Config.ssl_client.flags TYPE: string DOC_START Various flags modifying the use of SSL while proxying https:// URLs: DONT_VERIFY_PEER Accept certificates that fail verification. For refined control, see sslproxy_cert_error. NO_DEFAULT_CA Don't use the default CA list built in to OpenSSL. DOC_END NAME: sslproxy_cert_error IFDEF: USE_SSL DEFAULT: none LOC: Config.ssl_client.cert_error TYPE: acl_access @@ -1998,40 +2166,123 @@ validation errors will result in ERR_SECURE_CONNECT_FAIL error. acl BrokenButTrustedServers dstdomain example.com sslproxy_cert_error allow BrokenButTrustedServers sslproxy_cert_error deny all This clause only supports fast acl types. See http://wiki.squid-cache.org/SquidFaq/SquidAcl for details. Using slow acl types may result in server crashes Without this option, all server certificate validation errors terminate the transaction. Bypassing validation errors is dangerous because an error usually implies that the server cannot be trusted and the connection may be insecure. See also: sslproxy_flags and DONT_VERIFY_PEER. Default setting: sslproxy_cert_error deny all DOC_END +NAME: sslproxy_cert_sign +IFDEF: USE_SSL +DEFAULT: none +POSTSCRIPTUM: signUntrusted ssl::certUntrusted +POSTSCRIPTUM: signSelf ssl::certSelfSigned +POSTSCRIPTUM: signTrusted all +TYPE: sslproxy_cert_sign +LOC: Config.ssl_client.cert_sign +DOC_START + + sslproxy_cert_sign acl ... + + The following certificate signing algorithms are supported: + signTrusted + Sign using the configured CA certificate which is usually + placed in and trusted by end-user browsers. This is the + default for trusted origin server certificates. + signUntrusted + Sign to guarantee an X509_V_ERR_CERT_UNTRUSTED browser error. + This is the default for untrusted origin server certificates + that are not self-signed (see ssl::certUntrusted). + signSelf + Sign using a self-signed certificate with the right CN to + generate a X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT error in the + browser. This is the default for self-signed origin server + certificates (see ssl::certSelfSigned). + + This clause only supports fast acl types. + + When sslproxy_cert_sign acl(s) match, Squid uses the corresponding + signing algorithm to generate the certificate and ignores all + subsequent sslproxy_cert_sign options (the first match wins). If no + acl(s) match, the default signing algorithm is determined by errors + detected when obtaining and validating the origin server certificate. + + WARNING: SQUID_X509_V_ERR_DOMAIN_MISMATCH and ssl:certDomainMismatch can + be used with sslproxy_cert_adapt, but if and only if Squid is bumping a + CONNECT request that carries a domain name. In all other cases (CONNECT + to an IP address or an intercepted SSL connection), Squid cannot detect + the domain mismatch at certificate generation time when + bump-server-first is used. +DOC_END + +NAME: sslproxy_cert_adapt +IFDEF: USE_SSL +DEFAULT: none +TYPE: sslproxy_cert_adapt +LOC: Config.ssl_client.cert_adapt +DOC_START + + sslproxy_cert_adapt acl ... + + The following certificate adaptation algorithms are supported: + setValidAfter + Sets the "Not After" property to the "Not After" property of + the CA certificate used to sign generated certificates. + setValidBefore + Sets the "Not Before" property to the "Not Before" property of + the CA certificate used to sign generated certificates. + setCommonName or setCommonName{CN} + Sets Subject.CN property to the host name specified as a + CN parameter or, if no explicit CN parameter was specified, + extracted from the CONNECT request. It is a misconfiguration + to use setCommonName without an explicit parameter for + intercepted or tproxied SSL connections. + + This clause only supports fast acl types. + + Squid first groups sslproxy_cert_adapt options by adaptation algorithm. + Within a group, when sslproxy_cert_adapt acl(s) match, Squid uses the + corresponding adaptation algorithm to generate the certificate and + ignores all subsequent sslproxy_cert_adapt options in that algorithm's + group (i.e., the first match wins within each algorithm group). If no + acl(s) match, the default mimicking action takes place. + + WARNING: SQUID_X509_V_ERR_DOMAIN_MISMATCH and ssl:certDomainMismatch can + be used with sslproxy_cert_adapt, but if and only if Squid is bumping a + CONNECT request that carries a domain name. In all other cases (CONNECT + to an IP address or an intercepted SSL connection), Squid cannot detect + the domain mismatch at certificate generation time when + bump-server-first is used. +DOC_END + NAME: sslpassword_program IFDEF: USE_SSL DEFAULT: none LOC: Config.Program.ssl_password TYPE: string DOC_START Specify a program used for entering SSL key passphrases when using encrypted SSL certificate keys. If not specified keys must either be unencrypted, or Squid started with the -N option to allow it to query interactively for the passphrase. The key file name is given as argument to the program allowing selection of the right password if you have multiple encrypted keys. DOC_END COMMENT_START OPTIONS RELATING TO EXTERNAL SSL_CRTD ----------------------------------------------------------------------------- COMMENT_END @@ -3074,40 +3325,58 @@ case of chunked requests the chunked encoding metadata are not included [http::]>sh Received HTTP request headers size [http::] #include #include #include #include #include #include #include +#include #include "cf_gen_defines.cci" #define MAX_LINE 1024 /* longest configuration line */ #define _PATH_PARSER "cf_parser.cci" #define _PATH_SQUID_CONF "squid.conf.documented" #define _PATH_SQUID_CONF_SHORT "squid.conf.default" #define _PATH_CF_DEPEND "cf.data.depend" enum State { sSTART, s1, sDOC, sNOCOMMENT, sEXIT }; typedef std::list LineList; typedef std::list TypeDepList; typedef std::list EntryAliasList; class DefaultValues { public: DefaultValues() : preset(), if_none(), docs() {} ~DefaultValues() {} /// Default config lines to be defined before parsing the config files. LineList preset; /// Default config lines to parse if the directive has no prior settings. /// This is mutually exclusive with preset values. /// An error will be printed during build if they clash. LineList if_none; + /// Default config lines to parse and add to any prior settings. + LineList postscriptum; + /// Text description to use in documentation for the default. /// If unset the preset or if-none values will be displayed. LineList docs; }; class Entry { public: Entry(const char *str) : name(str), alias(),type(), loc(), defaults(), comment(), ifdef(), doc(), nocomment(), array_flag(0) {} ~Entry() {} std::string name; EntryAliasList alias; std::string type; std::string loc; DefaultValues defaults; std::string comment; @@ -134,40 +138,42 @@ class Type { public: Type(const char *str) : name(str) {} ~Type() {} std::string name; TypeDepList depend; }; typedef std::list TypeList; static const char WS[] = " \t\n"; static int gen_default(const EntryList &, std::ostream &); static void gen_parse(const EntryList &, std::ostream &); static void gen_dump(const EntryList &, std::ostream&); static void gen_free(const EntryList &, std::ostream&); static void gen_conf(const EntryList &, std::ostream&, bool verbose_output); static void gen_default_if_none(const EntryList &, std::ostream&); +static void gen_default_postscriptum(const EntryList &, std::ostream&); +static bool isDefined(const std::string &name); static void checkDepend(const std::string &directive, const char *name, const TypeList &types, const EntryList &entries) { for (TypeList::const_iterator t = types.begin(); t != types.end(); ++t) { if (t->name.compare(name) != 0) continue; for (TypeDepList::const_iterator dep = t->depend.begin(); dep != t->depend.end(); ++dep) { EntryList::const_iterator entry = entries.begin(); for (; entry != entries.end(); ++entry) { if (entry->name.compare(*dep) == 0) break; } if (entry == entries.end()) { std::cerr << "ERROR: '" << directive << "' (" << name << ") depends on '" << *dep << "'\n"; exit(1); } } return; } @@ -181,40 +187,41 @@ std::cerr << "Usage: " << program_name << " cf.data cf.data.depend\n"; exit(1); } int main(int argc, char *argv[]) { char *input_filename; const char *output_filename = _PATH_PARSER; const char *conf_filename = _PATH_SQUID_CONF; const char *conf_filename_short = _PATH_SQUID_CONF_SHORT; const char *type_depend; int linenum = 0; EntryList entries; TypeList types; enum State state; int rc = 0; char *ptr = NULL; char buff[MAX_LINE]; std::ifstream fp; + std::stack IFDEFS; if (argc != 3) usage(argv[0]); input_filename = argv[1]; type_depend = argv[2]; /*-------------------------------------------------------------------* * Parse type dependencies *-------------------------------------------------------------------*/ fp.open(type_depend, std::ifstream::in); if (fp.fail()) { std::cerr << "error while opening type dependencies file '" << type_depend << "': " << strerror(errno) << std::endl; exit(1); } while (fp.good()) { fp.getline(buff,MAX_LINE); const char *type = strtok(buff, WS); @@ -235,40 +242,55 @@ *-------------------------------------------------------------------*/ /* Open input file */ fp.open(input_filename, std::ifstream::in); if (fp.fail()) { std::cerr << "error while opening input file '" << input_filename << "': " << strerror(errno) << std::endl; exit(1); } state = sSTART; while (fp.getline(buff,MAX_LINE), fp.good() && state != sEXIT) { char *t; linenum++; if ((t = strchr(buff, '\n'))) *t = '\0'; + if(strncmp(buff, "IF ", 3) == 0) { + if ((ptr = strtok(buff + 3, WS)) == NULL) { + std::cerr << "Missing IF parameter on line" << linenum << std::endl; + exit(1); + } + IFDEFS.push(ptr); + continue; + } else if (strcmp(buff, "ENDIF") == 0) { + if (IFDEFS.size() == 0) { + std::cerr << "ENDIF without IF before on line " << linenum << std::endl; + exit(1); + } + IFDEFS.pop(); + } + else if (!IFDEFS.size() || isDefined(IFDEFS.top())) switch (state) { case sSTART: if ((strlen(buff) == 0) || (!strncmp(buff, "#", 1))) { /* ignore empty and comment lines */ (void) 0; } else if (!strncmp(buff, "NAME:", 5)) { char *name, *aliasname; if ((name = strtok(buff + 5, WS)) == NULL) { std::cerr << "Error in input file\n"; exit(1); } entries.push_back(name); while ((aliasname = strtok(NULL, WS)) != NULL) entries.back().alias.push_front(aliasname); @@ -297,40 +319,47 @@ ptr = buff + 8; while (isspace((unsigned char)*ptr)) ptr++; curr.comment = ptr; } else if (!strncmp(buff, "DEFAULT:", 8)) { ptr = buff + 8; while (isspace((unsigned char)*ptr)) ptr++; curr.defaults.preset.push_back(ptr); } else if (!strncmp(buff, "DEFAULT_IF_NONE:", 16)) { ptr = buff + 16; while (isspace((unsigned char)*ptr)) ptr++; curr.defaults.if_none.push_back(ptr); + } else if (!strncmp(buff, "POSTSCRIPTUM:", 13)) { + ptr = buff + 13; + + while (isspace((unsigned char)*ptr)) + ptr++; + + curr.defaults.postscriptum.push_back(ptr); } else if (!strncmp(buff, "DEFAULT_DOC:", 12)) { ptr = buff + 12; while (isspace((unsigned char)*ptr)) ptr++; curr.defaults.docs.push_back(ptr); } else if (!strncmp(buff, "LOC:", 4)) { if ((ptr = strtok(buff + 4, WS)) == NULL) { std::cerr << "Error on line " << linenum << std::endl; exit(1); } curr.loc = ptr; } else if (!strncmp(buff, "TYPE:", 5)) { if ((ptr = strtok(buff + 5, WS)) == NULL) { std::cerr << "Error on line " << linenum << std::endl; exit(1); } @@ -407,40 +436,42 @@ std::ofstream fout(output_filename,std::ostream::out); if (!fout.good()) { std::cerr << "error while opening output .c file '" << output_filename << "': " << strerror(errno) << std::endl; exit(1); } fout << "/*\n" << " * Generated automatically from " << input_filename << " by " << argv[0] << "\n" " *\n" " * Abstract: This file contains routines used to configure the\n" " * variables in the squid server.\n" " */\n" "\n"; rc = gen_default(entries, fout); gen_default_if_none(entries, fout); + gen_default_postscriptum(entries, fout); + gen_parse(entries, fout); gen_dump(entries, fout); gen_free(entries, fout); fout.close(); /* Open output x.conf file */ fout.open(conf_filename,std::ostream::out); if (!fout.good()) { std::cerr << "error while opening output conf file '" << output_filename << "': " << strerror(errno) << std::endl; exit(1); } gen_conf(entries, fout, 1); fout.close(); @@ -535,40 +566,70 @@ if (!entry->defaults.preset.empty()) { std::cerr << "ERROR: " << entry->name << " has preset defaults. DEFAULT_IF_NONE cannot be true." << std::endl; exit(1); } if (entry->ifdef.size()) fout << "#if " << entry->ifdef << std::endl; fout << " if (check_null_" << entry->type << "(" << entry->loc << ")) {" << std::endl; for (LineList::const_iterator l = entry->defaults.if_none.begin(); l != entry->defaults.if_none.end(); ++l) fout << " default_line(\"" << entry->name << " " << *l <<"\");" << std::endl; fout << " }" << std::endl; if (entry->ifdef.size()) fout << "#endif" << std::endl; } fout << "}" << std::endl << std::endl; } +/// append configuration options specified by POSTSCRIPTUM lines +static void +gen_default_postscriptum(const EntryList &head, std::ostream &fout) +{ + fout << "static void" << std::endl << + "defaults_postscriptum(void)" << std::endl << + "{" << std::endl; + + for (EntryList::const_iterator entry = head.begin(); entry != head.end(); ++entry) { + assert(entry->name.size()); + + if (!entry->loc.size()) + continue; + + if (entry->defaults.postscriptum.empty()) + continue; + + if (entry->ifdef.size()) + fout << "#if " << entry->ifdef << std::endl; + + for (LineList::const_iterator l = entry->defaults.postscriptum.begin(); l != entry->defaults.postscriptum.end(); ++l) + fout << " default_line(\"" << entry->name << " " << *l <<"\");" << std::endl; + + if (entry->ifdef.size()) + fout << "#endif" << std::endl; + } + + fout << "}" << std::endl << std::endl; +} + void Entry::genParseAlias(const std::string &aName, std::ostream &fout) const { fout << " if (!strcmp(token, \"" << aName << "\")) {" << std::endl; fout << " "; if (type.compare("obsolete") == 0) { fout << "debugs(0, DBG_CRITICAL, \"ERROR: Directive '" << aName << "' is obsolete.\");\n"; for (LineList::const_iterator l = doc.begin(); l != doc.end(); ++l) { // offset line to strip initial whitespace tab byte fout << " debugs(0, opt_parse_cfg_only?0:1, \"" << aName << " : " << &(*l)[1] << "\");" << std::endl; } fout << " parse_obsolete(token);"; } else if (!loc.size() || loc.compare("none") == 0) { fout << "parse_" << type << "();"; } else { fout << "parse_" << type << "(&" << loc << (array_flag ? "[0]" : "") << ");"; } fout << std::endl; fout << " return 1;" << std::endl; fout << " };" << std::endl; === modified file 'src/client_side.cc' --- src/client_side.cc 2012-06-18 15:43:05 +0000 +++ src/client_side.cc 2012-07-12 13:37:54 +0000 @@ -88,58 +88,61 @@ #include "auth/UserRequest.h" #endif #include "anyp/PortCfg.h" #include "base/Subscription.h" #include "base/TextException.h" #include "ChunkedCodingParser.h" #include "client_side.h" #include "client_side_reply.h" #include "client_side_request.h" #if USE_DELAY_POOLS #include "ClientInfo.h" #endif #include "ClientRequestContext.h" #include "clientStream.h" #include "comm.h" #include "comm/Connection.h" #include "CommCalls.h" #include "comm/Loops.h" #include "comm/Write.h" #include "comm/TcpAcceptor.h" +#include "errorpage.h" #include "eui/Config.h" #include "fde.h" +#include "forward.h" #include "HttpHdrContRange.h" #include "HttpReply.h" #include "HttpRequest.h" #include "ident/Config.h" #include "ident/Ident.h" #include "ipc/FdNotes.h" #include "ipc/StartListening.h" #include "MemBuf.h" #include "MemObject.h" #include "rfc1738.h" #include "StatCounters.h" #include "StatHist.h" #include "SquidTime.h" #if USE_SSL #include "ssl/context_storage.h" #include "ssl/helper.h" +#include "ssl/ServerBump.h" #include "ssl/support.h" #include "ssl/gadgets.h" #endif #if USE_SSL_CRTD #include "ssl/crtd_message.h" #include "ssl/certificate_db.h" #endif #include "Store.h" #include "TimeOrTag.h" #if HAVE_LIMITS #include #endif #if LINGERING_CLOSE #define comm_close comm_lingering_close #endif /// dials clientListenerConnectionOpened call class ListeningStartedDialer: public CallDialer, public Ipc::StartListeningCb @@ -793,40 +796,44 @@ return cbdataReferenceValid(this) && // XXX: checking "this" in a method Comm::IsConnOpen(clientConnection) && !fd_table[clientConnection->fd].closing(); } ConnStateData::~ConnStateData() { assert(this != NULL); debugs(33, 3, HERE << clientConnection); if (isOpen()) debugs(33, 1, "BUG: ConnStateData did not close " << clientConnection); if (!flags.swanSang) debugs(33, 1, "BUG: ConnStateData was not destroyed properly; " << clientConnection); cbdataReferenceDone(port); if (bodyPipe != NULL) stopProducingFor(bodyPipe, false); + +#if USE_SSL + delete sslServerBump; +#endif } /** * clientSetKeepaliveFlag() sets request->flags.proxy_keepalive. * This is the client-side persistent connection flag. We need * to set this relatively early in the request processing * to handle hacks for broken servers and clients. */ static void clientSetKeepaliveFlag(ClientHttpRequest * http) { HttpRequest *request = http->request; debugs(33, 3, "clientSetKeepaliveFlag: http_ver = " << request->http_ver.major << "." << request->http_ver.minor); debugs(33, 3, "clientSetKeepaliveFlag: method = " << RequestMethodStr(request->method)); // TODO: move to HttpRequest::hdrCacheInit, just like HttpReply. request->flags.proxy_keepalive = request->persistent() ? 1 : 0; @@ -2426,137 +2433,234 @@ HTTP_BAD_REQUEST, METHOD_NONE, NULL, clientConnection->remote, NULL, NULL, NULL); context->registerWithConn(); context->pullData(); } void ConnStateData::clientAfterReadingRequests() { // Were we expecting to read more request body from half-closed connection? if (mayNeedToReadMoreBody() && commIsHalfClosed(clientConnection->fd)) { debugs(33, 3, HERE << "truncated body: closing half-closed " << clientConnection); clientConnection->close(); return; } if (flags.readMore) readSomeData(); } +void +ConnStateData::quitAfterError(HttpRequest *request) +{ + // From HTTP p.o.v., we do not have to close after every error detected + // at the client-side, but many such errors do require closure and the + // client-side code is bad at handling errors so we play it safe. + if (request) + request->flags.proxy_keepalive = 0; + flags.readMore = false; + debugs(33,4, HERE << "Will close after error: " << clientConnection); +} + +#if USE_SSL +bool ConnStateData::serveDelayedError(ClientSocketContext *context) +{ + ClientHttpRequest *http = context->http; + + if (!sslServerBump) + return false; + + assert(sslServerBump->entry); + // Did we create an error entry while processing CONNECT? + if (!sslServerBump->entry->isEmpty()) { + quitAfterError(http->request); + + // Get the saved error entry and send it to the client by replacing the + // ClientHttpRequest store entry with it. + clientStreamNode *node = context->getClientReplyContext(); + clientReplyContext *repContext = dynamic_cast(node->data.getRaw()); + assert(repContext); + debugs(33, 5, "Responding with delated error for " << http->uri); + repContext->setReplyToStoreEntry(sslServerBump->entry); + + // save the original request for logging purposes + if (!context->http->al.request) + context->http->al.request = HTTPMSGLOCK(http->request); + + // Get error details from the fake certificate-peeking request. + http->request->detailError(sslServerBump->request->errType, sslServerBump->request->errDetail); + context->pullData(); + return true; + } + + // In bump-server-first mode, we have not necessarily seen the intended + // server name at certificate-peeking time. Check for domain mismatch now, + // when we can extract the intended name from the bumped HTTP request. + if (sslServerBump->serverCert.get()) { + HttpRequest *request = http->request; + if (!Ssl::checkX509ServerValidity(sslServerBump->serverCert.get(), request->GetHost())) { + debugs(33, 2, "SQUID_X509_V_ERR_DOMAIN_MISMATCH: Certificate " << + "does not match domainname " << request->GetHost()); + + ACLFilledChecklist check(Config.ssl_client.cert_error, request, dash_str); + check.sslErrors = new Ssl::Errors(SQUID_X509_V_ERR_DOMAIN_MISMATCH); + if (Comm::IsConnOpen(pinning.serverConnection)) + check.fd(pinning.serverConnection->fd); + const bool allowDomainMismatch = + check.fastCheck() == ACCESS_ALLOWED; + delete check.sslErrors; + check.sslErrors = NULL; + + if (!allowDomainMismatch) { + quitAfterError(request); + + clientStreamNode *node = context->getClientReplyContext(); + clientReplyContext *repContext = dynamic_cast(node->data.getRaw()); + assert (repContext); + + // Fill the server IP and hostname for error page generation. + HttpRequest::Pointer const & peekerRequest = sslServerBump->request; + request->hier.note(peekerRequest->hier.tcpServer, request->GetHost()); + + // Create an error object and fill it + ErrorState *err = new ErrorState(ERR_SECURE_CONNECT_FAIL, HTTP_SERVICE_UNAVAILABLE, request); + err->src_addr = clientConnection->remote; + Ssl::ErrorDetail *errDetail = new Ssl::ErrorDetail( + SQUID_X509_V_ERR_DOMAIN_MISMATCH, + sslServerBump->serverCert.get(), NULL); + err->detail = errDetail; + // Save the original request for logging purposes. + if (!context->http->al.request) + context->http->al.request = HTTPMSGLOCK(request); + repContext->setReplyToError(request->method, err); + assert(context->http->out.offset == 0); + context->pullData(); + return true; + } + } + } + + return false; +} +#endif // USE_SSL + static void clientProcessRequest(ConnStateData *conn, HttpParser *hp, ClientSocketContext *context, const HttpRequestMethod& method, HttpVersion http_ver) { ClientHttpRequest *http = context->http; HttpRequest *request = NULL; bool notedUseOfBuffer = false; bool chunked = false; bool mustReplyToOptions = false; bool unsupportedTe = false; bool expectBody = false; /* We have an initial client stream in place should it be needed */ /* setup our private context */ context->registerWithConn(); if (context->flags.parsed_ok == 0) { clientStreamNode *node = context->getClientReplyContext(); debugs(33, 2, "clientProcessRequest: Invalid Request"); + conn->quitAfterError(NULL); // setLogUri should called before repContext->setReplyToError setLogUri(http, http->uri, true); clientReplyContext *repContext = dynamic_cast(node->data.getRaw()); assert (repContext); switch (hp->request_parse_status) { case HTTP_HEADER_TOO_LARGE: repContext->setReplyToError(ERR_TOO_BIG, HTTP_BAD_REQUEST, method, http->uri, conn->clientConnection->remote, NULL, conn->in.buf, NULL); break; case HTTP_METHOD_NOT_ALLOWED: repContext->setReplyToError(ERR_UNSUP_REQ, HTTP_METHOD_NOT_ALLOWED, method, http->uri, conn->clientConnection->remote, NULL, conn->in.buf, NULL); break; default: repContext->setReplyToError(ERR_INVALID_REQ, hp->request_parse_status, method, http->uri, conn->clientConnection->remote, NULL, conn->in.buf, NULL); } assert(context->http->out.offset == 0); context->pullData(); - conn->flags.readMore = false; goto finish; } if ((request = HttpRequest::CreateFromUrlAndMethod(http->uri, method)) == NULL) { clientStreamNode *node = context->getClientReplyContext(); debugs(33, 5, "Invalid URL: " << http->uri); + conn->quitAfterError(request); // setLogUri should called before repContext->setReplyToError setLogUri(http, http->uri, true); clientReplyContext *repContext = dynamic_cast(node->data.getRaw()); assert (repContext); repContext->setReplyToError(ERR_INVALID_URL, HTTP_BAD_REQUEST, method, http->uri, conn->clientConnection->remote, NULL, NULL, NULL); assert(context->http->out.offset == 0); context->pullData(); - conn->flags.readMore = false; goto finish; } /* RFC 2616 section 10.5.6 : handle unsupported HTTP versions cleanly. */ /* We currently only accept 0.9, 1.0, 1.1 */ if ( (http_ver.major == 0 && http_ver.minor != 9) || (http_ver.major == 1 && http_ver.minor > 1 ) || (http_ver.major > 1) ) { clientStreamNode *node = context->getClientReplyContext(); debugs(33, 5, "Unsupported HTTP version discovered. :\n" << HttpParserHdrBuf(hp)); + conn->quitAfterError(request); // setLogUri should called before repContext->setReplyToError setLogUri(http, http->uri, true); clientReplyContext *repContext = dynamic_cast(node->data.getRaw()); assert (repContext); repContext->setReplyToError(ERR_UNSUP_HTTPVERSION, HTTP_HTTP_VERSION_NOT_SUPPORTED, method, http->uri, conn->clientConnection->remote, NULL, HttpParserHdrBuf(hp), NULL); assert(context->http->out.offset == 0); context->pullData(); - conn->flags.readMore = false; goto finish; } /* compile headers */ /* we should skip request line! */ /* XXX should actually know the damned buffer size here */ if (http_ver.major >= 1 && !request->parseHeader(HttpParserHdrBuf(hp), HttpParserHdrSz(hp))) { clientStreamNode *node = context->getClientReplyContext(); debugs(33, 5, "Failed to parse request headers:\n" << HttpParserHdrBuf(hp)); + conn->quitAfterError(request); // setLogUri should called before repContext->setReplyToError setLogUri(http, http->uri, true); clientReplyContext *repContext = dynamic_cast(node->data.getRaw()); assert (repContext); repContext->setReplyToError(ERR_INVALID_REQ, HTTP_BAD_REQUEST, method, http->uri, conn->clientConnection->remote, NULL, NULL, NULL); assert(context->http->out.offset == 0); context->pullData(); - conn->flags.readMore = false; goto finish; } request->clientConnectionManager = conn; request->flags.accelerated = http->flags.accel; request->flags.sslBumped = conn->switchedToHttps(); + request->flags.canRePin = request->flags.sslBumped && conn->pinning.pinned; request->flags.ignore_cc = conn->port->ignore_cc; - request->flags.no_direct = request->flags.accelerated ? !conn->port->allow_direct : 0; + // TODO: decouple http->flags.accel from request->flags.sslBumped + request->flags.no_direct = (request->flags.accelerated && !request->flags.sslBumped) ? + !conn->port->allow_direct : 0; #if USE_AUTH if (request->flags.sslBumped) { if (conn->auth_user_request != NULL) request->auth_user_request = conn->auth_user_request; } #endif /** \par * If transparent or interception mode is working clone the transparent and interception flags * from the port settings to the request. */ if (http->clientConnection != NULL) { request->flags.intercepted = ((http->clientConnection->flags & COMM_INTERCEPTION) != 0); request->flags.spoof_client_ip = ((http->clientConnection->flags & COMM_TRANSPARENT) != 0 ) ; } if (internalCheck(request->urlpath.termedBuf())) { if (internalHostnameIs(request->GetHost()) && request->port == getMyPort()) { http->flags.internal = 1; @@ -2583,111 +2687,116 @@ request->my_addr = conn->clientConnection->local; request->myportname = conn->port->name; request->http_ver = http_ver; // Link this HttpRequest to ConnStateData relatively early so the following complex handling can use it // TODO: this effectively obsoletes a lot of conn->FOO copying. That needs cleaning up later. request->clientConnectionManager = conn; if (request->header.chunked()) { chunked = true; } else if (request->header.has(HDR_TRANSFER_ENCODING)) { const String te = request->header.getList(HDR_TRANSFER_ENCODING); // HTTP/1.1 requires chunking to be the last encoding if there is one unsupportedTe = te.size() && te != "identity"; } // else implied identity coding mustReplyToOptions = (method == METHOD_OPTIONS) && (request->header.getInt64(HDR_MAX_FORWARDS) == 0); if (!urlCheckRequest(request) || mustReplyToOptions || unsupportedTe) { clientStreamNode *node = context->getClientReplyContext(); + conn->quitAfterError(request); clientReplyContext *repContext = dynamic_cast(node->data.getRaw()); assert (repContext); repContext->setReplyToError(ERR_UNSUP_REQ, HTTP_NOT_IMPLEMENTED, request->method, NULL, conn->clientConnection->remote, request, NULL, NULL); assert(context->http->out.offset == 0); context->pullData(); - conn->flags.readMore = false; goto finish; } if (!chunked && !clientIsContentLengthValid(request)) { clientStreamNode *node = context->getClientReplyContext(); clientReplyContext *repContext = dynamic_cast(node->data.getRaw()); assert (repContext); + conn->quitAfterError(request); repContext->setReplyToError(ERR_INVALID_REQ, HTTP_LENGTH_REQUIRED, request->method, NULL, conn->clientConnection->remote, request, NULL, NULL); assert(context->http->out.offset == 0); context->pullData(); - conn->flags.readMore = false; goto finish; } if (request->header.has(HDR_EXPECT)) { const String expect = request->header.getList(HDR_EXPECT); const bool supportedExpect = (expect.caseCmp("100-continue") == 0); if (!supportedExpect) { clientStreamNode *node = context->getClientReplyContext(); clientReplyContext *repContext = dynamic_cast(node->data.getRaw()); assert (repContext); + conn->quitAfterError(request); repContext->setReplyToError(ERR_INVALID_REQ, HTTP_EXPECTATION_FAILED, request->method, http->uri, conn->clientConnection->remote, request, NULL, NULL); assert(context->http->out.offset == 0); context->pullData(); - conn->flags.readMore = false; goto finish; } } http->request = HTTPMSGLOCK(request); clientSetKeepaliveFlag(http); // Let tunneling code be fully responsible for CONNECT requests if (http->request->method == METHOD_CONNECT) { context->mayUseConnection(true); conn->flags.readMore = false; } +#if USE_SSL + if (conn->switchedToHttps() && conn->serveDelayedError(context)) + goto finish; +#endif + /* Do we expect a request-body? */ expectBody = chunked || request->content_length > 0; if (!context->mayUseConnection() && expectBody) { request->body_pipe = conn->expectRequestBody( chunked ? -1 : request->content_length); // consume header early so that body pipe gets just the body connNoteUseOfBuffer(conn, http->req_sz); notedUseOfBuffer = true; /* Is it too large? */ if (!chunked && // if chunked, we will check as we accumulate clientIsRequestBodyTooLargeForPolicy(request->content_length)) { clientStreamNode *node = context->getClientReplyContext(); clientReplyContext *repContext = dynamic_cast(node->data.getRaw()); assert (repContext); + conn->quitAfterError(request); repContext->setReplyToError(ERR_TOO_BIG, HTTP_REQUEST_ENTITY_TOO_LARGE, METHOD_NONE, NULL, conn->clientConnection->remote, http->request, NULL, NULL); assert(context->http->out.offset == 0); context->pullData(); - conn->flags.readMore = false; goto finish; } // We may stop producing, comm_close, and/or call setReplyToError() // below, so quit on errors to avoid http->doCallouts() if (!conn->handleRequestBodyData()) goto finish; if (!request->body_pipe->productionEnded()) { debugs(33, 5, HERE << "need more request body"); context->mayUseConnection(true); assert(conn->flags.readMore); } } http->calloutContext = new ClientRequestContext(http); http->doCallouts(); finish: @@ -3409,209 +3518,459 @@ if (client_cert != NULL) { debugs(83, 3, "clientNegotiateSSL: FD " << fd << " client certificate: subject: " << X509_NAME_oneline(X509_get_subject_name(client_cert), 0, 0)); debugs(83, 3, "clientNegotiateSSL: FD " << fd << " client certificate: issuer: " << X509_NAME_oneline(X509_get_issuer_name(client_cert), 0, 0)); X509_free(client_cert); } else { debugs(83, 5, "clientNegotiateSSL: FD " << fd << " has no certificate."); } conn->readSomeData(); } +/** + * If SSL_CTX is given, starts reading the SSL handshake. + * Otherwise, calls switchToHttps to generate a dynamic SSL_CTX. + */ +static void +httpsEstablish(ConnStateData *connState, SSL_CTX *sslContext, Ssl::BumpMode bumpMode) +{ + SSL *ssl = NULL; + assert(connState); + const Comm::ConnectionPointer &details = connState->clientConnection; + + if (sslContext && !(ssl = httpsCreate(details, sslContext))) + return; + + typedef CommCbMemFunT TimeoutDialer; + AsyncCall::Pointer timeoutCall = JobCallback(33, 5, TimeoutDialer, + connState, ConnStateData::requestTimeout); + commSetConnTimeout(details, Config.Timeout.request, timeoutCall); + + if (ssl) + Comm::SetSelect(details->fd, COMM_SELECT_READ, clientNegotiateSSL, connState, 0); + else { + char buf[MAX_IPSTRLEN]; + assert(bumpMode != Ssl::bumpNone && bumpMode != Ssl::bumpEnd); + HttpRequest *fakeRequest = new HttpRequest; + fakeRequest->SetHost(details->local.NtoA(buf, sizeof(buf))); + fakeRequest->port = details->local.GetPort(); + fakeRequest->clientConnectionManager = connState; + fakeRequest->client_addr = connState->clientConnection->remote; +#if FOLLOW_X_FORWARDED_FOR + fakeRequest->indirect_client_addr = connState->clientConnection->remote; +#endif + fakeRequest->my_addr = connState->clientConnection->local; + fakeRequest->flags.spoof_client_ip = ((connState->clientConnection->flags & COMM_TRANSPARENT) != 0 ) ; + fakeRequest->flags.intercepted = ((connState->clientConnection->flags & COMM_INTERCEPTION) != 0); + debugs(33, 4, HERE << details << " try to generate a Dynamic SSL CTX"); + connState->switchToHttps(fakeRequest, bumpMode); + } +} + +/** + * A callback function to use with the ACLFilledChecklist callback. + * In the case of ACCES_ALLOWED answer initializes a bumped SSL connection, + * else reverts the connection to tunnel mode. + */ +static void +httpsSslBumpAccessCheckDone(allow_t answer, void *data) +{ + ConnStateData *connState = (ConnStateData *) data; + + // if the connection is closed or closing, just return. + if (!connState->isOpen()) + return; + + // Require both a match and a positive bump mode to work around exceptional + // cases where ACL code may return ACCESS_ALLOWED with zero answer.kind. + if (answer == ACCESS_ALLOWED && answer.kind != Ssl::bumpNone) { + debugs(33, 2, HERE << "sslBump needed for " << connState->clientConnection); + connState->sslBumpMode = static_cast(answer.kind); + httpsEstablish(connState, NULL, (Ssl::BumpMode)answer.kind); + } else { + debugs(33, 2, HERE << "sslBump not needed for " << connState->clientConnection); + connState->sslBumpMode = Ssl::bumpNone; + + // fake a CONNECT request to force connState to tunnel + static char ip[MAX_IPSTRLEN]; + static char reqStr[MAX_IPSTRLEN + 80]; + connState->clientConnection->local.NtoA(ip, sizeof(ip)); + snprintf(reqStr, sizeof(reqStr), "CONNECT %s:%d HTTP/1.1\r\nHost: %s\r\n\r\n", ip, connState->clientConnection->local.GetPort(), ip); + bool ret = connState->handleReadData(reqStr, strlen(reqStr)); + if (ret) + ret = connState->clientParseRequests(); + + if (!ret) { + debugs(33, 2, HERE << "Failed to start fake CONNECT request for ssl bumped connection: " << connState->clientConnection); + connState->clientConnection->close(); + } + } +} + /** handle a new HTTPS connection */ static void httpsAccept(const CommAcceptCbParams ¶ms) { AnyP::PortCfg *s = static_cast(params.data); if (params.flag != COMM_OK) { // Its possible the call was still queued when the client disconnected debugs(33, 2, "httpsAccept: " << s->listenConn << ": accept failure: " << xstrerr(params.xerrno)); return; } - SSL_CTX *sslContext = s->staticSslContext.get(); - SSL *ssl = NULL; - if (!(ssl = httpsCreate(params.conn, sslContext))) - return; - debugs(33, 4, HERE << params.conn << " accepted, starting SSL negotiation."); fd_note(params.conn->fd, "client https connect"); if (s->tcp_keepalive.enabled) { commSetTcpKeepalive(params.conn->fd, s->tcp_keepalive.idle, s->tcp_keepalive.interval, s->tcp_keepalive.timeout); } incoming_sockets_accepted++; // Socket is ready, setup the connection manager to start using it ConnStateData *connState = connStateCreate(params.conn, s); - typedef CommCbMemFunT TimeoutDialer; - AsyncCall::Pointer timeoutCall = JobCallback(33, 5, - TimeoutDialer, connState, ConnStateData::requestTimeout); - commSetConnTimeout(params.conn, Config.Timeout.request, timeoutCall); + if (s->sslBump) { + debugs(33, 5, "httpsAccept: accept transparent connection: " << params.conn); - Comm::SetSelect(params.conn->fd, COMM_SELECT_READ, clientNegotiateSSL, connState, 0); + if (!Config.accessList.ssl_bump) { + httpsSslBumpAccessCheckDone(ACCESS_DENIED, connState); + return; + } + + // Create a fake HTTP request for ssl_bump ACL check, + // using tproxy/intercept provided destination IP and port. + HttpRequest *request = new HttpRequest(); + static char ip[MAX_IPSTRLEN]; + assert(params.conn->flags & (COMM_TRANSPARENT | COMM_INTERCEPTION)); + request->SetHost(params.conn->local.NtoA(ip, sizeof(ip))); + request->port = params.conn->local.GetPort(); + request->myportname = s->name; + + ACLFilledChecklist *acl_checklist = new ACLFilledChecklist(Config.accessList.ssl_bump, request, NULL); + acl_checklist->src_addr = params.conn->remote; + acl_checklist->my_addr = s->s; + acl_checklist->nonBlockingCheck(httpsSslBumpAccessCheckDone, connState); + return; + } else { + SSL_CTX *sslContext = s->staticSslContext.get(); + httpsEstablish(connState, sslContext, Ssl::bumpNone); + } } void ConnStateData::sslCrtdHandleReplyWrapper(void *data, char *reply) { ConnStateData * state_data = (ConnStateData *)(data); state_data->sslCrtdHandleReply(reply); } void ConnStateData::sslCrtdHandleReply(const char * reply) { if (!reply) { debugs(1, 1, HERE << "\"ssl_crtd\" helper return reply"); } else { Ssl::CrtdMessage reply_message; if (reply_message.parse(reply, strlen(reply)) != Ssl::CrtdMessage::OK) { - debugs(33, 5, HERE << "Reply from ssl_crtd for " << sslHostName << " is incorrect"); + debugs(33, 5, HERE << "Reply from ssl_crtd for " << sslConnectHostOrIp << " is incorrect"); } else { if (reply_message.getCode() != "OK") { - debugs(33, 5, HERE << "Certificate for " << sslHostName << " cannot be generated. ssl_crtd response: " << reply_message.getBody()); + debugs(33, 5, HERE << "Certificate for " << sslConnectHostOrIp << " cannot be generated. ssl_crtd response: " << reply_message.getBody()); } else { - debugs(33, 5, HERE << "Certificate for " << sslHostName << " was successfully recieved from ssl_crtd"); + debugs(33, 5, HERE << "Certificate for " << sslConnectHostOrIp << " was successfully recieved from ssl_crtd"); SSL_CTX *ctx = Ssl::generateSslContextUsingPkeyAndCertFromMemory(reply_message.getBody().c_str()); getSslContextDone(ctx, true); return; } } } getSslContextDone(NULL); } -bool +void ConnStateData::buildSslCertGenerationParams(Ssl::CertificateProperties &certProperties) +{ + certProperties.commonName = sslCommonName.defined() ? sslCommonName.termedBuf() : sslConnectHostOrIp.termedBuf(); + + // fake certificate adaptation requires bump-server-first mode + if (!sslServerBump) { + assert(port->signingCert.get()); + certProperties.signWithX509.resetAndLock(port->signingCert.get()); + if (port->signPkey.get()) + certProperties.signWithPkey.resetAndLock(port->signPkey.get()); + certProperties.signAlgorithm = Ssl::algSignTrusted; + return; + } + + // In case of an error while connecting to the secure server, use a fake + // trusted certificate, with no mimicked fields and no adaptation + // algorithms. There is nothing we can mimic so we want to minimize the + // number of warnings the user will have to see to get to the error page. + assert(sslServerBump->entry); + if (sslServerBump->entry->isEmpty()) { + if (X509 *mimicCert = sslServerBump->serverCert.get()) + certProperties.mimicCert.resetAndLock(mimicCert); + + ACLFilledChecklist checklist(NULL, sslServerBump->request, + clientConnection != NULL ? clientConnection->rfc931 : dash_str); + checklist.conn(this); + checklist.sslErrors = cbdataReference(sslServerBump->sslErrors); + + for (sslproxy_cert_adapt *ca = Config.ssl_client.cert_adapt; ca != NULL; ca = ca->next) { + // If the algorithm already set, then ignore it. + if ((ca->alg == Ssl::algSetCommonName && certProperties.setCommonName) || + (ca->alg == Ssl::algSetValidAfter && certProperties.setValidAfter) || + (ca->alg == Ssl::algSetValidBefore && certProperties.setValidBefore) ) + continue; + + if (ca->aclList && checklist.fastCheck(ca->aclList) == ACCESS_ALLOWED) { + const char *alg = Ssl::CertAdaptAlgorithmStr[ca->alg]; + const char *param = ca->param; + + // For parameterless CN adaptation, use hostname from the + // CONNECT request. + if (ca->alg == Ssl::algSetCommonName) { + if (!param) + param = sslConnectHostOrIp.termedBuf(); + certProperties.commonName = param; + certProperties.setCommonName = true; + } + else if(ca->alg == Ssl::algSetValidAfter) + certProperties.setValidAfter = true; + else if(ca->alg == Ssl::algSetValidBefore) + certProperties.setValidBefore = true; + + debugs(33, 5, HERE << "Matches certificate adaptation aglorithm: " << + alg << " param: " << (param ? param : "-")); + } + } + + certProperties.signAlgorithm = Ssl::algSignEnd; + for (sslproxy_cert_sign *sg = Config.ssl_client.cert_sign; sg != NULL; sg = sg->next) { + if (sg->aclList && checklist.fastCheck(sg->aclList) == ACCESS_ALLOWED) { + certProperties.signAlgorithm = (Ssl::CertSignAlgorithm)sg->alg; + break; + } + } + } else {// if (!sslServerBump->entry->isEmpty()) + // Use trusted certificate for a Squid-generated error + // or the user would have to add a security exception + // just to see the error page. We will close the connection + // so that the trust is not extended to non-Squid content. + certProperties.signAlgorithm = Ssl::algSignTrusted; + } + + assert(certProperties.signAlgorithm != Ssl::algSignEnd); + + if (certProperties.signAlgorithm == Ssl::algSignUntrusted) { + assert(port->untrustedSigningCert.get()); + certProperties.signWithX509.resetAndLock(port->untrustedSigningCert.get()); + certProperties.signWithPkey.resetAndLock(port->untrustedSignPkey.get()); + } + else { + assert(port->signingCert.get()); + certProperties.signWithX509.resetAndLock(port->signingCert.get()); + + if (port->signPkey.get()) + certProperties.signWithPkey.resetAndLock(port->signPkey.get()); + } + signAlgorithm = certProperties.signAlgorithm; +} + +void ConnStateData::getSslContextStart() { - char const * host = sslHostName.termedBuf(); - if (port->generateHostCertificates && host && strcmp(host, "") != 0) { - debugs(33, 5, HERE << "Finding SSL certificate for " << host << " in cache"); + assert(areAllContextsForThisConnection()); + freeAllContexts(); + /* careful: freeAllContexts() above frees request, host, etc. */ + + if (port->generateHostCertificates) { + Ssl::CertificateProperties certProperties; + buildSslCertGenerationParams(certProperties); + sslBumpCertKey = certProperties.dbKey().c_str(); + assert(sslBumpCertKey.defined() && sslBumpCertKey[0] != '\0'); + + debugs(33, 5, HERE << "Finding SSL certificate for " << sslBumpCertKey << " in cache"); Ssl::LocalContextStorage & ssl_ctx_cache(Ssl::TheGlobalContextStorage.getLocalStorage(port->s)); - SSL_CTX * dynCtx = ssl_ctx_cache.find(host); + SSL_CTX * dynCtx = ssl_ctx_cache.find(sslBumpCertKey.termedBuf()); if (dynCtx) { - debugs(33, 5, HERE << "SSL certificate for " << host << " have found in cache"); - if (Ssl::verifySslCertificateDate(dynCtx)) { - debugs(33, 5, HERE << "Cached SSL certificate for " << host << " is valid"); - return getSslContextDone(dynCtx); + debugs(33, 5, HERE << "SSL certificate for " << sslBumpCertKey << " have found in cache"); + if (Ssl::verifySslCertificate(dynCtx, certProperties)) { + debugs(33, 5, HERE << "Cached SSL certificate for " << sslBumpCertKey << " is valid"); + getSslContextDone(dynCtx); + return; } else { - debugs(33, 5, HERE << "Cached SSL certificate for " << host << " is out of date. Delete this certificate from cache"); - ssl_ctx_cache.remove(host); + debugs(33, 5, HERE << "Cached SSL certificate for " << sslBumpCertKey << " is out of date. Delete this certificate from cache"); + ssl_ctx_cache.remove(sslBumpCertKey.termedBuf()); } } else { - debugs(33, 5, HERE << "SSL certificate for " << host << " haven't found in cache"); + debugs(33, 5, HERE << "SSL certificate for " << sslBumpCertKey << " haven't found in cache"); } #if USE_SSL_CRTD - debugs(33, 5, HERE << "Generating SSL certificate for " << host << " using ssl_crtd."); + try { + debugs(33, 5, HERE << "Generating SSL certificate for " << certProperties.commonName << " using ssl_crtd."); Ssl::CrtdMessage request_message; request_message.setCode(Ssl::CrtdMessage::code_new_certificate); - Ssl::CrtdMessage::BodyParams map; - map.insert(std::make_pair(Ssl::CrtdMessage::param_host, host)); - std::string bufferToWrite; - Ssl::writeCertAndPrivateKeyToMemory(port->signingCert, port->signPkey, bufferToWrite); - request_message.composeBody(map, bufferToWrite); + request_message.composeRequest(certProperties); + debugs(33, 5, HERE << "SSL crtd request: " << request_message.compose().c_str()); Ssl::Helper::GetInstance()->sslSubmit(request_message, sslCrtdHandleReplyWrapper, this); - return true; -#else - debugs(33, 5, HERE << "Generating SSL certificate for " << host); - dynCtx = Ssl::generateSslContext(host, port->signingCert, port->signPkey); - return getSslContextDone(dynCtx, true); -#endif //USE_SSL_CRTD + return; + } + catch (const std::exception &e) { + debugs(33, DBG_IMPORTANT, "ERROR: Failed to compose ssl_crtd " << + "request for " << certProperties.commonName << + " certificate: " << e.what() << "; will now block to " << + "generate that certificate."); + // fall through to do blocking in-process generation. + } +#endif // USE_SSL_CRTD + + debugs(33, 5, HERE << "Generating SSL certificate for " << certProperties.commonName); + dynCtx = Ssl::generateSslContext(certProperties); + getSslContextDone(dynCtx, true); + return; } - return getSslContextDone(NULL); + getSslContextDone(NULL); } -bool +void ConnStateData::getSslContextDone(SSL_CTX * sslContext, bool isNew) { // Try to add generated ssl context to storage. if (port->generateHostCertificates && isNew) { - Ssl::addChainToSslContext(sslContext, port->certsToChain.get()); + if (signAlgorithm == Ssl::algSignTrusted) + Ssl::addChainToSslContext(sslContext, port->certsToChain.get()); + //else it is self-signed or untrusted do not attrach any certificate Ssl::LocalContextStorage & ssl_ctx_cache(Ssl::TheGlobalContextStorage.getLocalStorage(port->s)); - if (sslContext && sslHostName != "") { - if (!ssl_ctx_cache.add(sslHostName.termedBuf(), sslContext)) { + assert(sslBumpCertKey.defined() && sslBumpCertKey[0] != '\0'); + if (sslContext) { + if (!ssl_ctx_cache.add(sslBumpCertKey.termedBuf(), sslContext)) { // If it is not in storage delete after using. Else storage deleted it. fd_table[clientConnection->fd].dynamicSslContext = sslContext; } } else { - debugs(33, 2, HERE << "Failed to generate SSL cert for " << sslHostName); + debugs(33, 2, HERE << "Failed to generate SSL cert for " << sslConnectHostOrIp); } } // If generated ssl context = NULL, try to use static ssl context. if (!sslContext) { if (!port->staticSslContext) { debugs(83, 1, "Closing SSL " << clientConnection->remote << " as lacking SSL context"); clientConnection->close(); - return false; + return; } else { debugs(33, 5, HERE << "Using static ssl context."); sslContext = port->staticSslContext.get(); } } SSL *ssl = NULL; if (!(ssl = httpsCreate(clientConnection, sslContext))) - return false; + return; // commSetConnTimeout() was called for this request before we switched. // Disable the client read handler until peer selection is complete Comm::SetSelect(clientConnection->fd, COMM_SELECT_READ, NULL, NULL, 0); Comm::SetSelect(clientConnection->fd, COMM_SELECT_READ, clientNegotiateSSL, this, 0); switchedToHttps_ = true; - return true; } -bool -ConnStateData::switchToHttps(const char *host) +void +ConnStateData::switchToHttps(HttpRequest *request, Ssl::BumpMode bumpServerMode) { assert(!switchedToHttps_); - sslHostName = host; - - //HTTPMSGLOCK(currentobject->http->request); - assert(areAllContextsForThisConnection()); - freeAllContexts(); - //currentobject->connIsFinished(); + sslConnectHostOrIp = request->GetHost(); + sslCommonName = request->GetHost(); // We are going to read new request flags.readMore = true; debugs(33, 5, HERE << "converting " << clientConnection << " to SSL"); - return getSslContextStart(); + // If sslServerBump is set, then we have decided to deny CONNECT + // and now want to switch to SSL to send the error to the client + // without even peeking at the origin server certificate. + if (bumpServerMode == Ssl::bumpServerFirst && !sslServerBump) { + request->flags.sslPeek = 1; + sslServerBump = new Ssl::ServerBump(request); + + // will call httpsPeeked() with certificate and connection, eventually + FwdState::fwdStart(clientConnection, sslServerBump->entry, sslServerBump->request); + return; + } + + // otherwise, use sslConnectHostOrIp + getSslContextStart(); +} + +void +ConnStateData::httpsPeeked(Comm::ConnectionPointer serverConnection) +{ + Must(sslServerBump != NULL); + + if (Comm::IsConnOpen(serverConnection)) { + SSL *ssl = fd_table[serverConnection->fd].ssl; + assert(ssl); + Ssl::X509_Pointer serverCert(SSL_get_peer_certificate(ssl)); + assert(serverCert.get() != NULL); + sslCommonName = Ssl::CommonHostName(serverCert.get()); + debugs(33, 5, HERE << "HTTPS server CN: " << sslCommonName << + " bumped: " << *serverConnection); + + pinConnection(serverConnection, NULL, NULL, false); + + debugs(33, 5, HERE << "bumped HTTPS server: " << sslConnectHostOrIp); + } else { + debugs(33, 5, HERE << "Error while bumping: " << sslConnectHostOrIp); + Ip::Address intendedDest; + intendedDest = sslConnectHostOrIp.termedBuf(); + const bool isConnectRequest = !port->spoof_client_ip && !port->intercepted; + + // Squid serves its own error page and closes, so we want + // a CN that causes no additional browser errors. Possible + // only when bumping CONNECT with a user-typed address. + if (intendedDest.IsAnyAddr() || isConnectRequest) + sslCommonName = sslConnectHostOrIp; + else if (sslServerBump->serverCert.get()) + sslCommonName = Ssl::CommonHostName(sslServerBump->serverCert.get()); + + // copy error detail from bump-server-first request to CONNECT request + if (currentobject != NULL && currentobject->http != NULL && currentobject->http->request) + currentobject->http->request->detailError(sslServerBump->request->errType, sslServerBump->request->errDetail); + } + + getSslContextStart(); } #endif /* USE_SSL */ /// check FD after clientHttp[s]ConnectionOpened, adjust HttpSockets as needed static bool OpenedHttpSocket(const Comm::ConnectionPointer &c, const Ipc::FdNoteId portType) { if (!Comm::IsConnOpen(c)) { Must(NHttpSockets > 0); // we tried to open some --NHttpSockets; // there will be fewer sockets than planned Must(HttpSockets[NHttpSockets] < 0); // no extra fds received if (!NHttpSockets) // we could not open any listen sockets at all fatalf("Unable to open %s",FdNote(portType)); return false; } return true; } @@ -3677,40 +4036,56 @@ #if USE_SSL static void clientHttpsConnectionsOpen(void) { AnyP::PortCfg *s; for (s = Config.Sockaddr.https; s; s = s->next) { if (MAXTCPLISTENPORTS == NHttpSockets) { debugs(1, 1, "Ignoring 'https_port' lines exceeding the limit."); debugs(1, 1, "The limit is " << MAXTCPLISTENPORTS << " HTTPS ports."); continue; } if (!s->staticSslContext) { debugs(1, 1, "Ignoring https_port " << s->s << " due to SSL initialization failure."); continue; } + // TODO: merge with similar code in clientHttpConnectionsOpen() + if (s->sslBump && !Config.accessList.ssl_bump) { + debugs(33, DBG_IMPORTANT, "WARNING: No ssl_bump configured. Disabling ssl-bump on " << s->protocol << "_port " << s->s); + s->sslBump = 0; + } + + if (s->sslBump && !s->staticSslContext && !s->generateHostCertificates) { + debugs(1, DBG_IMPORTANT, "Will not bump SSL at http_port " << s->s << " due to SSL initialization failure."); + s->sslBump = 0; + } + + if (s->sslBump) { + // Create ssl_ctx cache for this port. + Ssl::TheGlobalContextStorage.addLocalStorage(s->s, s->dynamicCertMemCacheSize == std::numeric_limits::max() ? 4194304 : s->dynamicCertMemCacheSize); + } + // Fill out a Comm::Connection which IPC will open as a listener for us s->listenConn = new Comm::Connection; s->listenConn->local = s->s; s->listenConn->flags = COMM_NONBLOCKING | (s->spoof_client_ip ? COMM_TRANSPARENT : 0) | (s->intercepted ? COMM_INTERCEPTION : 0); // setup the subscriptions such that new connections accepted by listenConn are handled by HTTPS typedef CommCbFunPtrCallT AcceptCall; RefCount subCall = commCbCall(5, 5, "httpsAccept", CommAcceptCbPtrFun(httpsAccept, s)); Subscription::Pointer sub = new CallSubscription(subCall); AsyncCall::Pointer listenCall = asyncCall(33, 2, "clientListenerConnectionOpened", ListeningStartedDialer(&clientListenerConnectionOpened, s, Ipc::fdnHttpsSocket, sub)); Ipc::StartListening(SOCK_STREAM, IPPROTO_TCP, s->listenConn, Ipc::fdnHttpsSocket, listenCall); HttpSockets[NHttpSockets++] = -1; } } #endif @@ -3854,41 +4229,45 @@ * the ident result on persistent connections... */ /* connection oriented auth also needs these two lines for it's operation. */ /* * Internal requests do not have a connection reference, because: A) their * byte count may be transformed before being applied to an outbound * connection B) they are internal - any limiting on them should be done on * the server end. */ if (conn != NULL) ch->conn(conn); /* unreferenced in FilledCheckList.cc */ return ch; } CBDATA_CLASS_INIT(ConnStateData); ConnStateData::ConnStateData() : AsyncJob("ConnStateData"), +#if USE_SSL + sslBumpMode(Ssl::bumpEnd), switchedToHttps_(false), + sslServerBump(NULL), +#endif stoppedSending_(NULL), stoppedReceiving_(NULL) { pinning.pinned = false; pinning.auth = false; } bool ConnStateData::transparent() const { return clientConnection != NULL && (clientConnection->flags & (COMM_TRANSPARENT|COMM_INTERCEPTION)); } bool ConnStateData::reading() const { return reader != NULL; } void @@ -4010,103 +4389,130 @@ } void ConnStateData::sendControlMsg(HttpControlMsg msg) { if (!isOpen()) { debugs(33, 3, HERE << "ignoring 1xx due to earlier closure"); return; } ClientSocketContext::Pointer context = getCurrentContext(); if (context != NULL) { context->writeControlMsg(msg); // will call msg.cbSuccess return; } debugs(33, 3, HERE << " closing due to missing context for 1xx"); clientConnection->close(); } -/* This is a comm call normally scheduled by comm_close() */ +/// Our close handler called by Comm when the pinned connection is closed void ConnStateData::clientPinnedConnectionClosed(const CommCloseCbParams &io) { - unpinConnection(); + // FwdState might repin a failed connection sooner than this close + // callback is called for the failed connection. + if (pinning.serverConnection == io.conn) { + pinning.closeHandler = NULL; // Comm unregisters handlers before calling + unpinConnection(); + } } void ConnStateData::pinConnection(const Comm::ConnectionPointer &pinServer, HttpRequest *request, struct peer *aPeer, bool auth) { char desc[FD_DESC_SZ]; if (Comm::IsConnOpen(pinning.serverConnection)) { if (pinning.serverConnection->fd == pinServer->fd) return; + } - unpinConnection(); // clears fields ready for re-use. Prevent close() scheduling our close handler. - pinning.serverConnection->close(); - } else - unpinConnection(); // clears fields ready for re-use. + unpinConnection(); // closes pinned connection, if any, and resets fields pinning.serverConnection = pinServer; - pinning.host = xstrdup(request->GetHost()); - pinning.port = request->port; + + debugs(33, 3, HERE << pinning.serverConnection); + + // when pinning an SSL bumped connection, the request may be NULL + const char *pinnedHost = "[unknown]"; + if (request) { + pinning.host = xstrdup(request->GetHost()); + pinning.port = request->port; + pinnedHost = pinning.host; + } else { + pinning.port = pinServer->remote.GetPort(); + } pinning.pinned = true; if (aPeer) pinning.peer = cbdataReference(aPeer); pinning.auth = auth; char stmp[MAX_IPSTRLEN]; snprintf(desc, FD_DESC_SZ, "%s pinned connection for %s (%d)", - (auth || !aPeer) ? request->GetHost() : aPeer->name, clientConnection->remote.ToURL(stmp,MAX_IPSTRLEN), clientConnection->fd); + (auth || !aPeer) ? pinnedHost : aPeer->name, + clientConnection->remote.ToURL(stmp,MAX_IPSTRLEN), + clientConnection->fd); fd_note(pinning.serverConnection->fd, desc); typedef CommCbMemFunT Dialer; pinning.closeHandler = JobCallback(33, 5, Dialer, this, ConnStateData::clientPinnedConnectionClosed); + // remember the pinned connection so that cb does not unpin a fresher one + typedef CommCloseCbParams Params; + Params ¶ms = GetCommParams(pinning.closeHandler); + params.conn = pinning.serverConnection; comm_add_close_handler(pinning.serverConnection->fd, pinning.closeHandler); } const Comm::ConnectionPointer ConnStateData::validatePinnedConnection(HttpRequest *request, const struct peer *aPeer) { + debugs(33, 7, HERE << pinning.serverConnection); + bool valid = true; if (!Comm::IsConnOpen(pinning.serverConnection)) valid = false; if (pinning.auth && request && strcasecmp(pinning.host, request->GetHost()) != 0) { valid = false; } if (request && pinning.port != request->port) { valid = false; } if (pinning.peer && !cbdataReferenceValid(pinning.peer)) { valid = false; } if (aPeer != pinning.peer) { valid = false; } if (!valid) { /* The pinning info is not safe, remove any pinning info */ unpinConnection(); } return pinning.serverConnection; } void ConnStateData::unpinConnection() { + debugs(33, 3, HERE << pinning.serverConnection); + if (pinning.peer) cbdataReferenceDone(pinning.peer); + if (Comm::IsConnOpen(pinning.serverConnection)) { if (pinning.closeHandler != NULL) { comm_remove_close_handler(pinning.serverConnection->fd, pinning.closeHandler); pinning.closeHandler = NULL; } /// also close the server side socket, we should not use it for any future requests... + // TODO: do not close if called from our close handler? pinning.serverConnection->close(); + } + safe_free(pinning.host); /* NOTE: pinning.pinned should be kept. This combined with fd == -1 at the end of a request indicates that the host * connection has gone away */ } === modified file 'src/client_side.h' --- src/client_side.h 2012-04-25 05:29:20 +0000 +++ src/client_side.h 2012-07-13 18:33:40 +0000 @@ -27,40 +27,43 @@ * You should have received a copy of the GNU General Public License * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. * */ #ifndef SQUID_CLIENTSIDE_H #define SQUID_CLIENTSIDE_H #if USE_AUTH #include "auth/UserRequest.h" #endif #include "base/AsyncJob.h" #include "BodyPipe.h" #include "comm.h" #include "CommCalls.h" #include "HttpControlMsg.h" #include "HttpParser.h" #include "RefCount.h" #include "StoreIOBuffer.h" +#if USE_SSL +#include "ssl/support.h" +#endif class ConnStateData; class ClientHttpRequest; class clientStreamNode; class ChunkedCodingParser; /** * Badly named. * This is in fact the processing context for a single HTTP request. * * Managing what has been done, and what happens next to the data buffer * holding what we hope is an HTTP request. * * Parsing is still a mess of global functions done in conjunction with the * real socket controller which generated ClientHttpRequest. * It also generates one of us and passes us control from there based on * the results of the parse. * * After that all the request interpretation and adaptation is in our scope. * Then finally the reply fetcher is created by this and we get the result @@ -143,41 +146,45 @@ static IOCB WroteControlMsg; void wroteControlMsg(const Comm::ConnectionPointer &conn, char *bufnotused, size_t size, comm_err_t errflag, int xerrno); private: CBDATA_CLASS(ClientSocketContext); void prepareReply(HttpReply * rep); void packChunk(const StoreIOBuffer &bodyData, MemBuf &mb); void packRange(StoreIOBuffer const &, MemBuf * mb); void deRegisterWithConn(); void doClose(); void initiateClose(const char *reason); AsyncCall::Pointer cbControlMsgSent; ///< notifies HttpControlMsg Source bool mayUseConnection_; /* This request may use the connection. Don't read anymore requests for now */ bool connRegistered_; }; class ConnectionDetail; - +#if USE_SSL +namespace Ssl { + class ServerBump; +} +#endif /** * Manages a connection to a client. * * Multiple requests (up to 2) can be pipelined. This object is responsible for managing * which one is currently being fulfilled and what happens to the queue if the current one * causes the client connection to be closed early. * * Act as a manager for the connection and passes data in buffer to the current parser. * the parser has ambiguous scope at present due to being made from global functions * I believe this object uses the parser to identify boundaries and kick off the * actual HTTP request handling objects (ClientSocketContext, ClientHttpRequest, HttpRequest) * * If the above can be confirmed accurate we can call this object PipelineManager or similar */ class ConnStateData : public BodyProducer, public HttpControlMsgSink { public: ConnStateData(); @@ -295,77 +302,111 @@ */ const Comm::ConnectionPointer validatePinnedConnection(HttpRequest *request, const struct peer *peer); /** * returts the pinned peer if exists, NULL otherwise */ struct peer *pinnedPeer() const {return pinning.peer;} bool pinnedAuth() const {return pinning.auth;} // pining related comm callbacks void clientPinnedConnectionClosed(const CommCloseCbParams &io); // comm callbacks void clientReadRequest(const CommIoCbParams &io); void connStateClosed(const CommCloseCbParams &io); void requestTimeout(const CommTimeoutCbParams ¶ms); // AsyncJob API virtual bool doneAll() const { return BodyProducer::doneAll() && false;} virtual void swanSong(); + /// Changes state so that we close the connection and quit after serving + /// the client-side-detected error response instead of getting stuck. + void quitAfterError(HttpRequest *request); // meant to be private + #if USE_SSL + /// called by FwdState when it is done bumping the server + void httpsPeeked(Comm::ConnectionPointer serverConnection); + /// Start to create dynamic SSL_CTX for host or uses static port SSL context. - bool getSslContextStart(); + void getSslContextStart(); /** * Done create dynamic ssl certificate. * * \param[in] isNew if generated certificate is new, so we need to add this certificate to storage. */ - bool getSslContextDone(SSL_CTX * sslContext, bool isNew = false); + void getSslContextDone(SSL_CTX * sslContext, bool isNew = false); /// Callback function. It is called when squid receive message from ssl_crtd. static void sslCrtdHandleReplyWrapper(void *data, char *reply); /// Proccess response from ssl_crtd. void sslCrtdHandleReply(const char * reply); - bool switchToHttps(const char *host); + void switchToHttps(HttpRequest *request, Ssl::BumpMode bumpServerMode); bool switchedToHttps() const { return switchedToHttps_; } + Ssl::ServerBump *serverBump() {return sslServerBump;} + inline void setServerBump(Ssl::ServerBump *srvBump) { + if (!sslServerBump) + sslServerBump = srvBump; + else + assert(sslServerBump == srvBump); + } + /// Fill the certAdaptParams with the required data for certificate adaptation + /// and create the key for storing/retrieve the certificate to/from the cache + void buildSslCertGenerationParams(Ssl::CertificateProperties &certProperties); + /// Called when the client sends the first request on a bumped connection. + /// Returns false if no [delayed] error should be written to the client. + /// Otherwise, writes the error to the client and returns true. Also checks + /// for SQUID_X509_V_ERR_DOMAIN_MISMATCH on bumped requests. + bool serveDelayedError(ClientSocketContext *context); + + Ssl::BumpMode sslBumpMode; ///< ssl_bump decision (Ssl::bumpEnd if n/a). + #else bool switchedToHttps() const { return false; } #endif protected: void startDechunkingRequest(); void finishDechunkingRequest(bool withSuccess); void abortChunkedRequestBody(const err_type error); err_type handleChunkedRequestBody(size_t &putSize); private: int connReadWasError(comm_err_t flag, int size, int xerrno); int connFinishedWithConn(int size); void clientAfterReadingRequests(); private: HttpParser parser_; // XXX: CBDATA plays with public/private and leaves the following 'private' fields all public... :( CBDATA_CLASS2(ConnStateData); +#if USE_SSL bool switchedToHttps_; + /// The SSL server host name appears in CONNECT request or the server ip address for the intercepted requests + String sslConnectHostOrIp; ///< The SSL server host name as passed in the CONNECT request + String sslCommonName; ///< CN name for SSL certificate generation + String sslBumpCertKey; ///< Key to use to store/retrieve generated certificate + + /// HTTPS server cert. fetching state for bump-ssl-server-first + Ssl::ServerBump *sslServerBump; + Ssl::CertSignAlgorithm signAlgorithm; ///< The signing algorithm to use +#endif /// the reason why we no longer write the response or nil const char *stoppedSending_; /// the reason why we no longer read the request or nil const char *stoppedReceiving_; - String sslHostName; ///< Host name for SSL certificate generation AsyncCall::Pointer reader; ///< set when we are reading BodyPipe::Pointer bodyPipe; // set when we are reading request body }; /* convenience class while splitting up body handling */ /* temporary existence only - on stack use expected */ void setLogUri(ClientHttpRequest * http, char const *uri, bool cleanUrl = false); const char *findTrailingHTTPVersion(const char *uriAndHTTPVersion, const char *end = NULL); #endif /* SQUID_CLIENTSIDE_H */ === modified file 'src/client_side_reply.cc' --- src/client_side_reply.cc 2012-05-12 03:21:00 +0000 +++ src/client_side_reply.cc 2012-05-12 16:02:59 +0000 @@ -96,55 +96,73 @@ * * This may be better placed in the clientStream logic, but it has not been * relocated there yet */ void clientReplyContext::setReplyToError( err_type err, http_status status, const HttpRequestMethod& method, char const *uri, Ip::Address &addr, HttpRequest * failedrequest, const char *unparsedrequest, #if USE_AUTH Auth::UserRequest::Pointer auth_user_request #else void* #endif ) { ErrorState *errstate = clientBuildError(err, status, uri, addr, failedrequest); if (unparsedrequest) errstate->request_hdrs = xstrdup(unparsedrequest); - if (status == HTTP_NOT_IMPLEMENTED && http->request) +#if USE_AUTH + errstate->auth_user_request = auth_user_request; +#endif + setReplyToError(method, errstate); +} + +void clientReplyContext::setReplyToError(const HttpRequestMethod& method, ErrorState *errstate) +{ + if (errstate->httpStatus == HTTP_NOT_IMPLEMENTED && http->request) /* prevent confusion over whether we default to persistent or not */ http->request->flags.proxy_keepalive = 0; http->al.http.code = errstate->httpStatus; createStoreEntry(method, request_flags()); -#if USE_AUTH - errstate->auth_user_request = auth_user_request; -#endif assert(errstate->callback_data == NULL); errorAppendEntry(http->storeEntry(), errstate); /* Now the caller reads to get this */ } +void clientReplyContext::setReplyToStoreEntry(StoreEntry *entry) +{ + entry->lock(); // removeClientStoreReference() unlocks + sc = storeClientListAdd(entry, this); +#if USE_DELAY_POOLS + sc->setDelayId(DelayId::DelayClient(http)); +#endif + reqofs = 0; + reqsize = 0; + flags.storelogiccomplete = 1; + http->storeEntry(entry); +} + void clientReplyContext::removeStoreReference(store_client ** scp, StoreEntry ** ep) { StoreEntry *e; store_client *sc_tmp = *scp; if ((e = *ep) != NULL) { *ep = NULL; storeUnregister(sc_tmp, e, this); *scp = NULL; e->unlock(); } } void clientReplyContext::removeClientStoreReference(store_client **scp, ClientHttpRequest *aHttpRequest) { StoreEntry *reference = aHttpRequest->storeEntry(); removeStoreReference(scp, &reference); @@ -1438,40 +1456,44 @@ /* Check whether we should send keep-alive */ if (!Config.onoff.error_pconns && reply->sline.status >= 400 && !request->flags.must_keepalive) { debugs(33, 3, "clientBuildReplyHeader: Error, don't keep-alive"); request->flags.proxy_keepalive = 0; } else if (!Config.onoff.client_pconns && !request->flags.must_keepalive) { debugs(33, 2, "clientBuildReplyHeader: Connection Keep-Alive not requested by admin or client"); request->flags.proxy_keepalive = 0; } else if (request->flags.proxy_keepalive && shutting_down) { debugs(88, 3, "clientBuildReplyHeader: Shutting down, don't keep-alive."); request->flags.proxy_keepalive = 0; } else if (request->flags.connection_auth && !reply->keep_alive) { debugs(33, 2, "clientBuildReplyHeader: Connection oriented auth but server side non-persistent"); request->flags.proxy_keepalive = 0; } else if (reply->bodySize(request->method) < 0 && !maySendChunkedReply) { debugs(88, 3, "clientBuildReplyHeader: can't keep-alive, unknown body size" ); request->flags.proxy_keepalive = 0; } else if (fdUsageHigh()&& !request->flags.must_keepalive) { debugs(88, 3, "clientBuildReplyHeader: Not many unused FDs, can't keep-alive"); request->flags.proxy_keepalive = 0; + } else if (request->flags.sslBumped && !reply->persistent()) { + // We do not really have to close, but we pretend we are a tunnel. + debugs(88, 3, "clientBuildReplyHeader: bumped reply forces close"); + request->flags.proxy_keepalive = 0; } // Decide if we send chunked reply if (maySendChunkedReply && request->flags.proxy_keepalive && reply->bodySize(request->method) < 0) { debugs(88, 3, "clientBuildReplyHeader: chunked reply"); request->flags.chunked_reply = 1; hdr->putStr(HDR_TRANSFER_ENCODING, "chunked"); } /* Append VIA */ if (Config.onoff.via) { LOCAL_ARRAY(char, bbuf, MAX_URL + 32); String strVia; hdr->getList(HDR_VIA, &strVia); snprintf(bbuf, MAX_URL + 32, "%d.%d %s", reply->sline.version.major, reply->sline.version.minor, ThisCache); === modified file 'src/client_side_reply.h' --- src/client_side_reply.h 2011-12-30 01:24:57 +0000 +++ src/client_side_reply.h 2012-06-20 14:27:31 +0000 @@ -54,46 +54,51 @@ clientReplyContext(ClientHttpRequest *); ~clientReplyContext(); void saveState(); void restoreState(); void purgeRequest (); void purgeRequestFindObjectToPurge(); void purgeDoMissPurge(); void purgeFoundGet(StoreEntry *newEntry); void purgeFoundHead(StoreEntry *newEntry); void purgeFoundObject(StoreEntry *entry); void sendClientUpstreamResponse(); void purgeDoPurgeGet(StoreEntry *entry); void purgeDoPurgeHead(StoreEntry *entry); void doGetMoreData(); void identifyStoreObject(); void identifyFoundObject(StoreEntry *entry); int storeOKTransferDone() const; int storeNotOKTransferDone() const; + /// replaces current response store entry with the given one + void setReplyToStoreEntry(StoreEntry *e); + /// builds error using clientBuildError() and calls setReplyToError() below void setReplyToError(err_type, http_status, const HttpRequestMethod&, char const *, Ip::Address &, HttpRequest *, const char *, #if USE_AUTH Auth::UserRequest::Pointer); #else void * unused); #endif + /// creates a store entry for the reply and appends err to it + void setReplyToError(const HttpRequestMethod& method, ErrorState *err); void createStoreEntry(const HttpRequestMethod& m, request_flags flags); void removeStoreReference(store_client ** scp, StoreEntry ** ep); void removeClientStoreReference(store_client **scp, ClientHttpRequest *http); void startError(ErrorState * err); void processExpired(); clientStream_status_t replyStatus(); void processMiss(); void traceReply(clientStreamNode * node); http_status purgeStatus; /* state variable - replace with class to handle storeentries at some point */ int lookingforstore; virtual void created (StoreEntry *newEntry); ClientHttpRequest *http; int headers_sz; store_client *sc; /* The store_client we're using */ StoreIOBuffer tempBuffer; /* For use in validating requests via IMS */ int old_reqsize; /* ... again, for the buffer */ === modified file 'src/client_side_request.cc' --- src/client_side_request.cc 2012-05-08 18:14:08 +0000 +++ src/client_side_request.cc 2012-07-12 14:50:22 +0000 @@ -49,66 +49,70 @@ #include "adaptation/AccessCheck.h" #include "adaptation/Answer.h" #include "adaptation/Iterator.h" #include "adaptation/Service.h" #if ICAP_CLIENT #include "adaptation/icap/History.h" #endif #endif #include "anyp/PortCfg.h" #if USE_AUTH #include "auth/UserRequest.h" #endif #include "clientStream.h" #include "client_side.h" #include "client_side_reply.h" #include "client_side_request.h" #include "ClientRequestContext.h" #include "comm/Connection.h" #include "comm/Write.h" #include "compat/inet_pton.h" +#include "errorpage.h" #include "fde.h" #include "format/Token.h" #include "HttpHdrCc.h" #include "HttpReply.h" #include "HttpRequest.h" #include "ip/QosConfig.h" #include "MemObject.h" #include "Store.h" #include "SquidTime.h" #include "wordlist.h" #include "err_detail_type.h" #if USE_SSL #include "ssl/support.h" +#include "ssl/ServerBump.h" #endif #if LINGERING_CLOSE #define comm_close comm_lingering_close #endif static const char *const crlf = "\r\n"; #if FOLLOW_X_FORWARDED_FOR static void clientFollowXForwardedForCheck(allow_t answer, void *data); #endif /* FOLLOW_X_FORWARDED_FOR */ +extern ErrorState *clientBuildError(err_type, http_status, char const *url, Ip::Address &, HttpRequest *); + CBDATA_CLASS_INIT(ClientRequestContext); void * ClientRequestContext::operator new (size_t size) { assert (size == sizeof(ClientRequestContext)); CBDATA_INIT_TYPE(ClientRequestContext); ClientRequestContext *result = cbdataAlloc(ClientRequestContext); return result; } void ClientRequestContext::operator delete (void *address) { ClientRequestContext *t = static_cast(address); cbdataFree(t); } /* Local functions */ /* other */ @@ -118,44 +122,45 @@ #endif static int clientHierarchical(ClientHttpRequest * http); static void clientInterpretRequestHeaders(ClientHttpRequest * http); static RH clientRedirectDoneWrapper; static void checkNoCacheDoneWrapper(allow_t, void *); extern "C" CSR clientGetMoreData; extern "C" CSS clientReplyStatus; extern "C" CSD clientReplyDetach; static void checkFailureRatio(err_type, hier_code); ClientRequestContext::~ClientRequestContext() { /* * Release our "lock" on our parent, ClientHttpRequest, if we * still have one */ if (http) cbdataReferenceDone(http); + delete error; debugs(85,3, HERE << this << " ClientRequestContext destructed"); } -ClientRequestContext::ClientRequestContext(ClientHttpRequest *anHttp) : http(cbdataReference(anHttp)), acl_checklist (NULL), redirect_state (REDIRECT_NONE) +ClientRequestContext::ClientRequestContext(ClientHttpRequest *anHttp) : http(cbdataReference(anHttp)), acl_checklist (NULL), redirect_state (REDIRECT_NONE), error(NULL), readNextRequest(false) { http_access_done = false; redirect_done = false; no_cache_done = false; interpreted_req_hdrs = false; #if USE_SSL sslBumpCheckDone = false; #endif debugs(85,3, HERE << this << " ClientRequestContext constructed"); } CBDATA_CLASS_INIT(ClientHttpRequest); void * ClientHttpRequest::operator new (size_t size) { assert (size == sizeof (ClientHttpRequest)); CBDATA_INIT_TYPE(ClientHttpRequest); ClientHttpRequest *result = cbdataAlloc(ClientHttpRequest); return result; @@ -165,41 +170,41 @@ ClientHttpRequest::operator delete (void *address) { ClientHttpRequest *t = static_cast(address); cbdataFree(t); } ClientHttpRequest::ClientHttpRequest(ConnStateData * aConn) : #if USE_ADAPTATION AsyncJob("ClientHttpRequest"), #endif loggingEntry_(NULL) { start_time = current_time; setConn(aConn); al.tcpClient = clientConnection = aConn->clientConnection; dlinkAdd(this, &active, &ClientActiveRequests); #if USE_ADAPTATION request_satisfaction_mode = false; #endif #if USE_SSL - sslBumpNeed = needUnknown; + sslBumpNeed_ = Ssl::bumpEnd; #endif } /* * returns true if client specified that the object must come from the cache * without contacting origin server */ bool ClientHttpRequest::onlyIfCached()const { assert(request); return request->cache_control && request->cache_control->onlyIfCached(); } /* * This function is designed to serve a fairly specific purpose. * Occasionally our vBNS-connected caches can talk to each other, but not * the rest of the world. Here we try to detect frequent failures which * make the cache unusable (e.g. DNS lookup and connect() failures). If @@ -791,60 +796,53 @@ } else if (!http->flags.accel) { /* Proxy authorisation needed */ status = HTTP_PROXY_AUTHENTICATION_REQUIRED; } else { /* WWW authorisation needed */ status = HTTP_UNAUTHORIZED; } #else // need auth, but not possible to do. status = HTTP_FORBIDDEN; #endif if (page_id == ERR_NONE) page_id = ERR_CACHE_ACCESS_DENIED; } else { status = HTTP_FORBIDDEN; if (page_id == ERR_NONE) page_id = ERR_ACCESS_DENIED; } - clientStreamNode *node = (clientStreamNode *)http->client_stream.tail->prev->data; - clientReplyContext *repContext = dynamic_cast(node->data.getRaw()); - assert (repContext); Ip::Address tmpnoaddr; tmpnoaddr.SetNoAddr(); - repContext->setReplyToError(page_id, status, - http->request->method, NULL, - http->getConn() != NULL ? http->getConn()->clientConnection->remote : tmpnoaddr, - http->request, - NULL, -#if USE_AUTH - http->getConn() != NULL && http->getConn()->auth_user_request != NULL ? - http->getConn()->auth_user_request : http->request->auth_user_request); -#else - NULL); -#endif - http->getConn()->flags.readMore = true; // resume any pipeline reads. - node = (clientStreamNode *)http->client_stream.tail->data; - clientStreamRead(node, http, node->readBuffer); - return; + error = clientBuildError(page_id, status, + NULL, + http->getConn() != NULL ? http->getConn()->clientConnection->remote : tmpnoaddr, + http->request + ); + + error->auth_user_request = + http->getConn() != NULL && http->getConn()->auth_user_request != NULL ? + http->getConn()->auth_user_request : http->request->auth_user_request; + + readNextRequest = true; } /* ACCESS_ALLOWED continues here ... */ safe_free(http->uri); http->uri = xstrdup(urlCanonical(http->request)); http->doCallouts(); } #if USE_ADAPTATION void ClientHttpRequest::noteAdaptationAclCheckDone(Adaptation::ServiceGroupPointer g) { debugs(93,3,HERE << this << " adaptationAclCheckDone called"); #if ICAP_CLIENT Adaptation::Icap::History::Pointer ih = request->icapHistory(); if (ih != NULL) { if (getConn() != NULL) { @@ -1262,71 +1260,101 @@ ClientRequestContext *calloutContext = (ClientRequestContext *) data; if (!calloutContext->httpStateIsValid()) return; calloutContext->checkNoCacheDone(answer); } void ClientRequestContext::checkNoCacheDone(const allow_t &answer) { acl_checklist = NULL; http->request->flags.cachable = (answer == ACCESS_ALLOWED); http->doCallouts(); } #if USE_SSL bool ClientRequestContext::sslBumpAccessCheck() { - if (http->request->method == METHOD_CONNECT && - Config.accessList.ssl_bump && http->getConn()->port->sslBump) { - debugs(85, 5, HERE << "SslBump possible, checking ACL"); + // If SSL connection tunneling or bumping decision has been made, obey it. + const Ssl::BumpMode bumpMode = http->getConn()->sslBumpMode; + if (bumpMode != Ssl::bumpEnd) { + debugs(85, 5, HERE << "SslBump already decided (" << bumpMode << + "), " << "ignoring ssl_bump for " << http->getConn()); + http->al.ssl.bumpMode = bumpMode; // inherited from bumped connection + return false; + } - ACLFilledChecklist *acl_checklist = clientAclChecklistCreate(Config.accessList.ssl_bump, http); - acl_checklist->nonBlockingCheck(sslBumpAccessCheckDoneWrapper, this); - return true; - } else { - http->sslBumpNeeded(false); + // If we have not decided yet, decide whether to bump now. + + // Bumping here can only start with a CONNECT request on a bumping port + // (bumping of intercepted SSL conns is decided before we get 1st request). + // We also do not bump redirected CONNECT requests. + if (http->request->method != METHOD_CONNECT || http->redirect.status || + !Config.accessList.ssl_bump || !http->getConn()->port->sslBump) { + http->al.ssl.bumpMode = Ssl::bumpEnd; // SslBump does not apply; log - + debugs(85, 5, HERE << "cannot SslBump this request"); return false; } + + // Do not bump during authentication: clients would not proxy-authenticate + // if we delay a 407 response and respond with 200 OK to CONNECT. + if (error && error->httpStatus == HTTP_PROXY_AUTHENTICATION_REQUIRED) { + http->al.ssl.bumpMode = Ssl::bumpEnd; // SslBump does not apply; log - + debugs(85, 5, HERE << "no SslBump during proxy authentication"); + return false; + } + + debugs(85, 5, HERE << "SslBump possible, checking ACL"); + + ACLFilledChecklist *acl_checklist = clientAclChecklistCreate(Config.accessList.ssl_bump, http); + acl_checklist->nonBlockingCheck(sslBumpAccessCheckDoneWrapper, this); + return true; } /** * A wrapper function to use the ClientRequestContext::sslBumpAccessCheckDone method * as ACLFilledChecklist callback */ static void sslBumpAccessCheckDoneWrapper(allow_t answer, void *data) { ClientRequestContext *calloutContext = static_cast(data); if (!calloutContext->httpStateIsValid()) return; - calloutContext->sslBumpAccessCheckDone(answer == ACCESS_ALLOWED); + calloutContext->sslBumpAccessCheckDone(answer); } void -ClientRequestContext::sslBumpAccessCheckDone(bool doSslBump) +ClientRequestContext::sslBumpAccessCheckDone(const allow_t &answer) { - http->sslBumpNeeded(doSslBump); + if (!httpStateIsValid()) + return; + + const Ssl::BumpMode bumpMode = answer == ACCESS_ALLOWED ? + static_cast(answer.kind) : Ssl::bumpNone; + http->sslBumpNeed(bumpMode); // for processRequest() to bump if needed + http->al.ssl.bumpMode = bumpMode; // for logging + http->doCallouts(); } #endif /* * Identify requests that do not go through the store and client side stream * and forward them to the appropriate location. All other requests, request * them. */ void ClientHttpRequest::processRequest() { debugs(85, 4, "clientProcessRequest: " << RequestMethodStr(request->method) << " '" << uri << "'"); if (request->method == METHOD_CONNECT && !redirect.status) { #if USE_SSL if (sslBumpNeeded()) { sslBumpStart(); return; } @@ -1340,93 +1368,93 @@ httpStart(); } void ClientHttpRequest::httpStart() { PROF_start(httpStart); logType = LOG_TAG_NONE; debugs(85, 4, "ClientHttpRequest::httpStart: " << Format::log_tags[logType] << " for '" << uri << "'"); /* no one should have touched this */ assert(out.offset == 0); /* Use the Stream Luke */ clientStreamNode *node = (clientStreamNode *)client_stream.tail->data; clientStreamRead(node, this, node->readBuffer); PROF_stop(httpStart); } #if USE_SSL -bool -ClientHttpRequest::sslBumpNeeded() const -{ - assert(sslBumpNeed != needUnknown); - return (sslBumpNeed == needConfirmed); -} - void -ClientHttpRequest::sslBumpNeeded(bool isNeeded) +ClientHttpRequest::sslBumpNeed(Ssl::BumpMode mode) { - debugs(83, 3, HERE << "sslBump required: "<< (isNeeded ? "Yes" : "No")); - sslBumpNeed = (isNeeded ? needConfirmed : needNot); + debugs(83, 3, HERE << "sslBump required: "<< Ssl::bumpMode(mode)); + sslBumpNeed_ = mode; } // called when comm_write has completed static void SslBumpEstablish(const Comm::ConnectionPointer &, char *, size_t, comm_err_t errflag, int, void *data) { ClientHttpRequest *r = static_cast(data); debugs(85, 5, HERE << "responded to CONNECT: " << r << " ? " << errflag); assert(r && cbdataReferenceValid(r)); r->sslBumpEstablish(errflag); } void ClientHttpRequest::sslBumpEstablish(comm_err_t errflag) { // Bail out quickly on COMM_ERR_CLOSING - close handlers will tidy up if (errflag == COMM_ERR_CLOSING) return; if (errflag) { debugs(85, 3, HERE << "CONNECT response failure in SslBump: " << errflag); getConn()->clientConnection->close(); return; } + // We lack HttpReply which logRequest() uses to log the status code. + // TODO: Use HttpReply instead of the "200 Connection established" string. + al.http.code = 200; + #if USE_AUTH // Preserve authentication info for the ssl-bumped request if (request->auth_user_request != NULL) getConn()->auth_user_request = request->auth_user_request; #endif - getConn()->switchToHttps(request->GetHost()); + + assert(sslBumpNeeded()); + getConn()->switchToHttps(request, sslBumpNeed_); } void ClientHttpRequest::sslBumpStart() { - debugs(85, 5, HERE << "Confirming CONNECT tunnel on FD " << getConn()->clientConnection); - // send an HTTP 200 response to kick client SSL negotiation - debugs(33, 7, HERE << "Confirming CONNECT tunnel on FD " << getConn()->clientConnection); + debugs(85, 5, HERE << "Confirming " << Ssl::bumpMode(sslBumpNeed_) << + "-bumped CONNECT tunnel on FD " << getConn()->clientConnection); + getConn()->sslBumpMode = sslBumpNeed_; + // send an HTTP 200 response to kick client SSL negotiation // TODO: Unify with tunnel.cc and add a Server(?) header static const char *const conn_established = "HTTP/1.1 200 Connection established\r\n\r\n"; AsyncCall::Pointer call = commCbCall(85, 5, "ClientSocketContext::sslBumpEstablish", CommIoCbPtrFun(&SslBumpEstablish, this)); Comm::Write(getConn()->clientConnection, conn_established, strlen(conn_established), call, NULL); } #endif bool ClientHttpRequest::gotEnough() const { /** TODO: should be querying the stream. */ int64_t contentLength = memObject()->getReply()->bodySize(request->method); assert(contentLength >= 0); if (out.offset < contentLength) return false; @@ -1478,133 +1506,167 @@ * longer valid, it should call cbdataReferenceDone() so that * ClientHttpRequest's reference count goes to zero and it will get * deleted. ClientHttpRequest will then delete ClientRequestContext. * * Note that we set the _done flags here before actually starting * the callout. This is strictly for convenience. */ extern tos_t aclMapTOS (acl_tos * head, ACLChecklist * ch); extern nfmark_t aclMapNfmark (acl_nfmark * head, ACLChecklist * ch); void ClientHttpRequest::doCallouts() { assert(calloutContext); /*Save the original request for logging purposes*/ if (!calloutContext->http->al.request) calloutContext->http->al.request = HTTPMSGLOCK(request); + if (!calloutContext->error) { // CVE-2009-0801: verify the Host: header is consistent with other known details. - if (!calloutContext->host_header_verify_done) { - debugs(83, 3, HERE << "Doing calloutContext->hostHeaderVerify()"); - calloutContext->host_header_verify_done = true; - calloutContext->hostHeaderVerify(); - return; - } + if (!calloutContext->host_header_verify_done) { + debugs(83, 3, HERE << "Doing calloutContext->hostHeaderVerify()"); + calloutContext->host_header_verify_done = true; + calloutContext->hostHeaderVerify(); + return; + } - if (!calloutContext->http_access_done) { - debugs(83, 3, HERE << "Doing calloutContext->clientAccessCheck()"); - calloutContext->http_access_done = true; - calloutContext->clientAccessCheck(); - return; - } + if (!calloutContext->http_access_done) { + debugs(83, 3, HERE << "Doing calloutContext->clientAccessCheck()"); + calloutContext->http_access_done = true; + calloutContext->clientAccessCheck(); + return; + } #if USE_ADAPTATION - if (!calloutContext->adaptation_acl_check_done) { - calloutContext->adaptation_acl_check_done = true; - if (Adaptation::AccessCheck::Start( + if (!calloutContext->adaptation_acl_check_done) { + calloutContext->adaptation_acl_check_done = true; + if (Adaptation::AccessCheck::Start( Adaptation::methodReqmod, Adaptation::pointPreCache, request, NULL, this)) - return; // will call callback - } + return; // will call callback + } #endif - if (!calloutContext->redirect_done) { - calloutContext->redirect_done = true; - assert(calloutContext->redirect_state == REDIRECT_NONE); + if (!calloutContext->redirect_done) { + calloutContext->redirect_done = true; + assert(calloutContext->redirect_state == REDIRECT_NONE); + + if (Config.Program.redirect) { + debugs(83, 3, HERE << "Doing calloutContext->clientRedirectStart()"); + calloutContext->redirect_state = REDIRECT_PENDING; + calloutContext->clientRedirectStart(); + return; + } + } - if (Config.Program.redirect) { - debugs(83, 3, HERE << "Doing calloutContext->clientRedirectStart()"); - calloutContext->redirect_state = REDIRECT_PENDING; - calloutContext->clientRedirectStart(); + if (!calloutContext->adapted_http_access_done) { + debugs(83, 3, HERE << "Doing calloutContext->clientAccessCheck2()"); + calloutContext->adapted_http_access_done = true; + calloutContext->clientAccessCheck2(); return; } - } - if (!calloutContext->adapted_http_access_done) { - debugs(83, 3, HERE << "Doing calloutContext->clientAccessCheck2()"); - calloutContext->adapted_http_access_done = true; - calloutContext->clientAccessCheck2(); - return; - } + if (!calloutContext->interpreted_req_hdrs) { + debugs(83, 3, HERE << "Doing clientInterpretRequestHeaders()"); + calloutContext->interpreted_req_hdrs = 1; + clientInterpretRequestHeaders(this); + } - if (!calloutContext->interpreted_req_hdrs) { - debugs(83, 3, HERE << "Doing clientInterpretRequestHeaders()"); - calloutContext->interpreted_req_hdrs = 1; - clientInterpretRequestHeaders(this); - } + if (!calloutContext->no_cache_done) { + calloutContext->no_cache_done = true; - if (!calloutContext->no_cache_done) { - calloutContext->no_cache_done = true; - - if (Config.accessList.noCache && request->flags.cachable) { - debugs(83, 3, HERE << "Doing calloutContext->checkNoCache()"); - calloutContext->checkNoCache(); - return; + if (Config.accessList.noCache && request->flags.cachable) { + debugs(83, 3, HERE << "Doing calloutContext->checkNoCache()"); + calloutContext->checkNoCache(); + return; + } } - } + } // if !calloutContext->error if (!calloutContext->tosToClientDone) { calloutContext->tosToClientDone = true; if (getConn() != NULL && Comm::IsConnOpen(getConn()->clientConnection)) { ACLFilledChecklist ch(NULL, request, NULL); ch.src_addr = request->client_addr; ch.my_addr = request->my_addr; tos_t tos = aclMapTOS(Ip::Qos::TheConfig.tosToClient, &ch); if (tos) Ip::Qos::setSockTos(getConn()->clientConnection, tos); } } if (!calloutContext->nfmarkToClientDone) { calloutContext->nfmarkToClientDone = true; if (getConn() != NULL && Comm::IsConnOpen(getConn()->clientConnection)) { ACLFilledChecklist ch(NULL, request, NULL); ch.src_addr = request->client_addr; ch.my_addr = request->my_addr; nfmark_t mark = aclMapNfmark(Ip::Qos::TheConfig.nfmarkToClient, &ch); if (mark) Ip::Qos::setSockNfmark(getConn()->clientConnection, mark); } } #if USE_SSL + // We need to check for SslBump even if the calloutContext->error is set + // because bumping may require delaying the error until after CONNECT. if (!calloutContext->sslBumpCheckDone) { calloutContext->sslBumpCheckDone = true; if (calloutContext->sslBumpAccessCheck()) return; /* else no ssl bump required*/ } #endif + if (calloutContext->error) { + const char *uri = urlCanonical(request); + StoreEntry *e= storeCreateEntry(uri, uri, request->flags, request->method); +#if USE_SSL + if (sslBumpNeeded()) { + // set final error but delay sending until we bump + Ssl::ServerBump *srvBump = new Ssl::ServerBump(request, e); + errorAppendEntry(e, calloutContext->error); + calloutContext->error = NULL; + getConn()->setServerBump(srvBump); + e->unlock(); + } else +#endif + { + // send the error to the client now + clientStreamNode *node = (clientStreamNode *)client_stream.tail->prev->data; + clientReplyContext *repContext = dynamic_cast(node->data.getRaw()); + assert (repContext); + repContext->setReplyToStoreEntry(e); + errorAppendEntry(e, calloutContext->error); + calloutContext->error = NULL; + if (calloutContext->readNextRequest) + getConn()->flags.readMore = true; // resume any pipeline reads. + node = (clientStreamNode *)client_stream.tail->data; + clientStreamRead(node, this, node->readBuffer); + e->unlock(); + return; + } + } + cbdataReferenceDone(calloutContext->http); delete calloutContext; calloutContext = NULL; #if HEADERS_LOG headersLog(0, 1, request->method, request); #endif debugs(83, 3, HERE << "calling processRequest()"); processRequest(); #if ICAP_CLIENT Adaptation::Icap::History::Pointer ih = request->icapHistory(); if (ih != NULL) ih->logType = logType; #endif } #if !_USE_INLINE_ #include "client_side_request.cci" @@ -1799,41 +1861,42 @@ const bool usedStore = storeEntry() && !storeEntry()->isEmpty(); const bool usedPipe = request->body_pipe != NULL && request->body_pipe->consumedSize() > 0; if (bypassable && !usedStore && !usedPipe) { debugs(85,3, HERE << "ICAP REQMOD callout failed, bypassing: " << calloutContext); if (calloutContext) doCallouts(); return; } debugs(85,3, HERE << "ICAP REQMOD callout failed, responding with error"); clientStreamNode *node = (clientStreamNode *)client_stream.tail->prev->data; clientReplyContext *repContext = dynamic_cast(node->data.getRaw()); assert(repContext); // The original author of the code also wanted to pass an errno to // setReplyToError, but it seems unlikely that the errno reflects the // true cause of the error at this point, so I did not pass it. - Ip::Address noAddr; - noAddr.SetNoAddr(); - ConnStateData * c = getConn(); - repContext->setReplyToError(ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR, - request->method, NULL, - (c != NULL ? c->clientConnection->remote : noAddr), request, NULL, + if (calloutContext) { + Ip::Address noAddr; + noAddr.SetNoAddr(); + ConnStateData * c = getConn(); + calloutContext->error = clientBuildError(ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR, + NULL, + c != NULL ? c->clientConnection->remote : noAddr, + request + ); #if USE_AUTH - (c != NULL && c->auth_user_request != NULL ? - c->auth_user_request : request->auth_user_request)); -#else - NULL); + calloutContext->error->auth_user_request = + c != NULL && c->auth_user_request != NULL ? c->auth_user_request : request->auth_user_request; #endif - - request->detailError(ERR_ICAP_FAILURE, errDetail); - c->flags.readMore = true; - c->expectNoForwarding(); - node = (clientStreamNode *)client_stream.tail->data; - clientStreamRead(node, this, node->readBuffer); + calloutContext->error->detailError(errDetail); + calloutContext->readNextRequest = true; + c->expectNoForwarding(); + doCallouts(); + } + //else if(calloutContext == NULL) is it possible? } #endif === modified file 'src/client_side_request.h' --- src/client_side_request.h 2012-01-20 18:55:04 +0000 +++ src/client_side_request.h 2012-07-01 15:32:13 +0000 @@ -138,48 +138,50 @@ ClientRequestContext *calloutContext; void doCallouts(); #if USE_ADAPTATION // AsyncJob virtual methods virtual bool doneAll() const { return Initiator::doneAll() && BodyConsumer::doneAll() && false; } #endif private: CBDATA_CLASS(ClientHttpRequest); int64_t maxReplyBodySize_; StoreEntry *entry_; StoreEntry *loggingEntry_; ConnStateData * conn_; #if USE_SSL - /// whether the request needs to be bumped - enum { needUnknown, needConfirmed, needNot } sslBumpNeed; + /// whether (and how) the request needs to be bumped + Ssl::BumpMode sslBumpNeed_; public: - /// return true if the request needs to be bumped - bool sslBumpNeeded() const; + /// returns raw sslBump mode value + Ssl::BumpMode sslBumpNeed() const { return sslBumpNeed_; } + /// returns true if and only if the request needs to be bumped + bool sslBumpNeeded() const { return sslBumpNeed_ == Ssl::bumpServerFirst || sslBumpNeed_ == Ssl::bumpClientFirst; } /// set the sslBumpNeeded state - void sslBumpNeeded(bool isNeeded); + void sslBumpNeed(Ssl::BumpMode mode); void sslBumpStart(); void sslBumpEstablish(comm_err_t errflag); #endif #if USE_ADAPTATION public: void startAdaptation(const Adaptation::ServiceGroupPointer &g); // private but exposed for ClientRequestContext void handleAdaptationFailure(int errDetail, bool bypassable = false); private: // Adaptation::Initiator API virtual void noteAdaptationAnswer(const Adaptation::Answer &answer); void handleAdaptedHeader(HttpMsg *msg); void handleAdaptationBlock(const Adaptation::Answer &answer); virtual void noteAdaptationAclCheckDone(Adaptation::ServiceGroupPointer group); // BodyConsumer API, called by BodyPipe === modified file 'src/errorpage.cc' --- src/errorpage.cc 2012-05-30 03:45:12 +0000 +++ src/errorpage.cc 2012-07-12 14:59:05 +0000 @@ -562,55 +562,55 @@ ErrorState::ErrorState(err_type t, http_status status, HttpRequest * req) : type(t), page_id(t), err_language(NULL), httpStatus(status), #if USE_AUTH auth_user_request (NULL), #endif request(NULL), url(NULL), xerrno(0), port(0), dnsError(), ttl(0), src_addr(), redirect_url(NULL), callback(NULL), callback_data(NULL), request_hdrs(NULL), - err_msg(NULL) + err_msg(NULL), #if USE_SSL - , detail(NULL) + detail(NULL), #endif + detailCode(ERR_DETAIL_NONE) { memset(&flags, 0, sizeof(flags)); memset(&ftp, 0, sizeof(ftp)); if (page_id >= ERR_MAX && ErrorDynamicPages.items[page_id - ERR_MAX]->page_redirect != HTTP_STATUS_NONE) httpStatus = ErrorDynamicPages.items[page_id - ERR_MAX]->page_redirect; if (req != NULL) { request = HTTPMSGLOCK(req); src_addr = req->client_addr; - request->detailError(type, ERR_DETAIL_NONE); } } void errorAppendEntry(StoreEntry * entry, ErrorState * err) { assert(entry->mem_obj != NULL); assert (entry->isEmpty()); debugs(4, 4, "Creating an error page for entry " << entry << " with errorstate " << err << " page id " << err->page_id); if (entry->store_status != STORE_PENDING) { debugs(4, 2, "Skipping error page due to store_status: " << entry->store_status); /* * If the entry is not STORE_PENDING, then no clients * care about it, and we don't need to generate an * error message */ assert(EBIT_TEST(entry->flags, ENTRY_ABORTED)); @@ -627,47 +627,40 @@ } entry->lock(); entry->buffer(); entry->replaceHttpReply( err->BuildHttpReply() ); EBIT_CLR(entry->flags, ENTRY_FWD_HDR_WAIT); entry->flush(); entry->complete(); entry->negativeCache(); entry->releaseRequest(); entry->unlock(); delete err; } void errorSend(const Comm::ConnectionPointer &conn, ErrorState * err) { HttpReply *rep; debugs(4, 3, HERE << conn << ", err=" << err); assert(Comm::IsConnOpen(conn)); - /* - * ugh, this is how we make sure error codes get back to - * the client side for logging and error tracking. - */ - - if (err->request) - err->request->detailError(err->type, err->xerrno); /* moved in front of errorBuildBuf @?@ */ err->flags.flag_cbdata = 1; rep = err->BuildHttpReply(); MemBuf *mb = rep->pack(); AsyncCall::Pointer call = commCbCall(78, 5, "errorSendComplete", CommIoCbPtrFun(&errorSendComplete, err)); Comm::Write(conn, mb, call); delete mb; delete rep; } /** \ingroup ErrorPageAPI * * Called by commHandleWrite() after data has been written * to the client socket. @@ -1203,40 +1196,56 @@ rep->header.delById(HDR_VARY); rep->header.putStr(HDR_VARY, "Accept-Language"); } /* add the Content-Language header according to RFC section 14.12 */ if (err_language) { rep->header.putStr(HDR_CONTENT_LANGUAGE, err_language); } else #endif /* USE_ERROR_LOCALES */ { /* default templates are in English */ /* language is known unless error_directory override used */ if (!Config.errorDirectory) rep->header.putStr(HDR_CONTENT_LANGUAGE, "en"); } rep->body.setMb(content); /* do not memBufClean() or delete the content, it was absorbed by httpBody */ } + // Make sure error codes get back to the client side for logging and + // error tracking. + if (request) { + int edc = ERR_DETAIL_NONE; // error detail code +#if USE_SSL + if (detail) + edc = detail->errorNo(); + else +#endif + if (detailCode) + edc = detailCode; + else + edc = xerrno; + request->detailError(type, edc); + } + return rep; } MemBuf * ErrorState::BuildContent() { const char *m = NULL; assert(page_id > ERR_NONE && page_id < error_page_count); #if USE_ERR_LOCALES ErrorPageFile *localeTmpl = NULL; /** error_directory option in squid.conf overrides translations. * Custom errors are always found either in error_directory or the templates directory. * Otherwise locate the Accept-Language header */ if (!Config.errorDirectory && page_id < ERR_MAX) { if (err_language && err_language != Config.errorDefaultLanguage) safe_free(err_language); === modified file 'src/errorpage.h' --- src/errorpage.h 2012-02-05 11:24:07 +0000 +++ src/errorpage.h 2012-07-13 10:02:38 +0000 @@ -23,40 +23,41 @@ * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. * * Copyright (c) 2003, Robert Collins */ #ifndef SQUID_ERRORPAGE_H #define SQUID_ERRORPAGE_H #include "squid-old.h" #if USE_AUTH #include "auth/UserRequest.h" #endif #include "cbdata.h" #include "comm/forward.h" +#include "err_detail_type.h" #include "ip/Address.h" #include "MemBuf.h" #if USE_SSL #include "ssl/ErrorDetail.h" #endif /** \defgroup ErrorPageAPI Error Pages API \ingroup Components \section ErrorPageStringCodes Error Page % codes for text insertion. * \verbatim a - User identity x B - URL with FTP %2f hack x c - Squid error code x d - seconds elapsed since request received x D - Error details x e - errno x E - strerror() x f - FTP request line x @@ -87,40 +88,43 @@ Z - Preformatted error message x \endverbatim */ class HttpReply; class MemBuf; /// \ingroup ErrorPageAPI class ErrorState { public: ErrorState(err_type type, http_status, HttpRequest * request); ErrorState(); // not implemented. ~ErrorState(); /** * Allocates and initializes an error response */ HttpReply *BuildHttpReply(void); + /// set error type-specific detail code + void detailError(int dCode) {detailCode = dCode;} + private: /** * Locates error page template to be used for this error * and constructs the HTML page content from it. */ MemBuf *BuildContent(void); /** * Convert the given template string into textual output * * \param text The string to be converted * \param allowRecursion Whether to convert codes which output may contain codes */ MemBuf *ConvertText(const char *text, bool allowRecursion); /** * Generates the Location: header value for a deny_info error page * to be used for this error. */ void DenyInfoLocation(const char *name, HttpRequest *request, MemBuf &result); @@ -165,40 +169,43 @@ void *callback_data; struct { unsigned int flag_cbdata:1; } flags; struct { wordlist *server_msg; char *request; char *reply; char *cwd_msg; MemBuf *listing; } ftp; char *request_hdrs; char *err_msg; /* Preformatted error message from the cache */ #if USE_SSL Ssl::ErrorDetail *detail; #endif + /// type-specific detail about the transaction error; + /// overwrites xerrno; overwritten by detail, if any. + int detailCode; private: CBDATA_CLASS2(ErrorState); }; /** \ingroup ErrorPageAPI * * This function finds the error messages formats, and stores * them in error_text[] * \par Global effects: * error_text[] - is modified */ SQUIDCEXTERN void errorInitialize(void); /// \ingroup ErrorPageAPI SQUIDCEXTERN void errorClean(void); /** * \ingroup ErrorPageAPI === modified file 'src/format/ByteCode.h' --- src/format/ByteCode.h 2011-11-18 07:48:25 +0000 +++ src/format/ByteCode.h 2012-06-20 15:28:40 +0000 @@ -173,37 +173,41 @@ LFT_ICAP_REQUEST_URI, LFT_ICAP_REQUEST_METHOD, LFT_ICAP_BYTES_SENT, LFT_ICAP_BYTES_READ, LFT_ICAP_BODY_BYTES_READ, LFT_ICAP_REQ_HEADER, LFT_ICAP_REQ_HEADER_ELEM, LFT_ICAP_REQ_ALL_HEADERS, LFT_ICAP_REP_HEADER, LFT_ICAP_REP_HEADER_ELEM, LFT_ICAP_REP_ALL_HEADERS, LFT_ICAP_TR_RESPONSE_TIME, LFT_ICAP_IO_TIME, LFT_ICAP_OUTCOME, LFT_ICAP_STATUS_CODE, #endif +#if USE_SSL + LFT_SSL_BUMP_MODE, +#endif + LFT_PERCENT /* special string cases for escaped chars */ } ByteCode_t; /// Quoting style for a format output. enum Quoting { LOG_QUOTE_NONE = 0, LOG_QUOTE_QUOTES, LOG_QUOTE_MIMEBLOB, LOG_QUOTE_URL, LOG_QUOTE_RAW }; extern const char *log_tags[]; } // namespace Format #endif /* _SQUID_FMT_BYTECODE_H */ === modified file 'src/format/Format.cc' --- src/format/Format.cc 2012-01-20 18:55:04 +0000 +++ src/format/Format.cc 2012-07-01 15:32:13 +0000 @@ -987,40 +987,49 @@ break; case LFT_IO_SIZE_TOTAL: outint = al->cache.requestSize + al->cache.replySize; doint = 1; break; case LFT_EXT_LOG: if (al->request) out = al->request->extacl_log.termedBuf(); quote = 1; break; case LFT_SEQUENCE_NUMBER: outoff = logSequenceNumber; dooff = 1; break; +#if USE_SSL + case LFT_SSL_BUMP_MODE: { + const Ssl::BumpMode mode = static_cast(al->ssl.bumpMode); + // for Ssl::bumpEnd, Ssl::bumpMode() returns NULL and we log '-' + out = Ssl::bumpMode(mode); + break; + } +#endif + case LFT_PERCENT: out = "%"; break; } if (dooff) { snprintf(tmp, sizeof(tmp), "%0*" PRId64, fmt->zero && fmt->widthMin >= 0 ? fmt->widthMin : 0, outoff); out = tmp; } else if (doint) { snprintf(tmp, sizeof(tmp), "%0*ld", fmt->zero && fmt->widthMin >= 0 ? fmt->widthMin : 0, outint); out = tmp; } if (out && *out) { if (quote || fmt->quote != LOG_QUOTE_NONE) { char *newout = NULL; int newfree = 0; === modified file 'src/format/Token.cc' --- src/format/Token.cc 2012-05-12 03:21:00 +0000 +++ src/format/Token.cc 2012-06-20 15:27:48 +0000 @@ -169,57 +169,65 @@ {"st", LFT_ICAP_BYTES_SENT}, {"h", LFT_ICAP_REQ_HEADER}, {"configTag != NULL; lte++) { debugs(46, 8, HERE << "compare tokens '" << lte->configTag << "' with '" << cur << "'"); if (strncmp(lte->configTag, cur, strlen(lte->configTag)) == 0) { type = lte->tokenType; label = lte->configTag; debugs(46, 7, HERE << "Found token '" << label << "'"); return cur + strlen(lte->configTag); } } return cur; } /* parses a single token. Returns the token length in characters, === modified file 'src/forward.cc' --- src/forward.cc 2012-06-12 07:01:02 +0000 +++ src/forward.cc 2012-06-22 14:59:05 +0000 @@ -18,64 +18,66 @@ * it under the terms of the GNU General Public License as published by * the Free Software Foundation; either version 2 of the License, or * (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. * */ #include "squid-old.h" #include "forward.h" #include "acl/FilledChecklist.h" #include "acl/Gadgets.h" +#include "anyp/PortCfg.h" #include "CacheManager.h" #include "comm/Connection.h" #include "comm/ConnOpener.h" #include "CommCalls.h" #include "comm/Loops.h" #include "event.h" #include "errorpage.h" #include "fde.h" #include "hier_code.h" #include "HttpReply.h" #include "HttpRequest.h" #include "ip/QosConfig.h" #include "MemObject.h" #include "pconn.h" #include "PeerSelectState.h" #include "SquidTime.h" #include "Store.h" #include "icmp/net_db.h" #include "ip/Intercept.h" #include "ip/tools.h" #include "mgr/Registration.h" #if USE_SSL #include "ssl/support.h" #include "ssl/ErrorDetail.h" +#include "ssl/ServerBump.h" #endif static PSC fwdPeerSelectionCompleteWrapper; static CLCB fwdServerClosedWrapper; #if USE_SSL static PF fwdNegotiateSSLWrapper; #endif static CNCB fwdConnectDoneWrapper; static OBJH fwdStats; #define MAX_FWD_STATS_IDX 9 static int FwdReplyCodes[MAX_FWD_STATS_IDX + 1][HTTP_INVALID_HEADER + 1]; static PconnPool *fwdPconnPool = new PconnPool("server-side"); CBDATA_CLASS_INIT(FwdState); void FwdState::abort(void* d) { @@ -107,54 +109,74 @@ // Called once, right after object creation, when it is safe to set self void FwdState::start(Pointer aSelf) { // Protect ourselves from being destroyed when the only Server pointing // to us is gone (while we expect to talk to more Servers later). // Once we set self, we are responsible for clearing it when we do not // expect to talk to any servers. self = aSelf; // refcounted // We hope that either the store entry aborts or peer is selected. // Otherwise we are going to leak our object. entry->registerAbort(FwdState::abort, this); // Bug 3243: CVE 2009-0801 // Bypass of browser same-origin access control in intercepted communication // To resolve this we must force DIRECT and only to the original client destination. const bool isIntercepted = request && !request->flags.redirected && (request->flags.intercepted || request->flags.spoof_client_ip); const bool useOriginalDst = Config.onoff.client_dst_passthru || (request && !request->flags.hostVerified); if (isIntercepted && useOriginalDst) { - Comm::ConnectionPointer p = new Comm::Connection(); - p->remote = clientConn->local; - p->peerType = ORIGINAL_DST; - getOutgoingAddress(request, p); - serverDestinations.push_back(p); - + selectPeerForIntercepted(); // destination "found". continue with the forwarding. startConnectionOrFail(); } else { // do full route options selection peerSelect(&serverDestinations, request, entry, fwdPeerSelectionCompleteWrapper, this); } } +/// bypasses peerSelect() when dealing with intercepted requests +void +FwdState::selectPeerForIntercepted() +{ + // use pinned connection if available + Comm::ConnectionPointer p; + if (ConnStateData *client = request->pinnedConnection()) + p = client->validatePinnedConnection(request, NULL); + + if (p != NULL && Comm::IsConnOpen(p)) { + debugs(17, 3, HERE << "reusing a pinned conn: " << *p); + /* duplicate peerSelectPinned() effects */ + p->peerType = PINNED; + entry->ping_status = PING_DONE; /* Skip ICP */ + } else { + p = new Comm::Connection(); + p->peerType = ORIGINAL_DST; + p->remote = clientConn->local; + getOutgoingAddress(request, p); + debugs(17, 3, HERE << "opening a new conn: " << *p); + } + + serverDestinations.push_back(p); +} + void FwdState::completed() { if (flags.forward_completed == 1) { debugs(17, 1, HERE << "FwdState::completed called on a completed request! Bad!"); return; } flags.forward_completed = 1; if (EBIT_TEST(entry->flags, ENTRY_ABORTED)) { debugs(17, 3, HERE << "entry aborted"); return ; } #if URL_CHECKSUM_DEBUG entry->mem_obj->checkUrlChecksum(); #endif @@ -285,75 +307,76 @@ /* NOTREACHED */ } void FwdState::startConnectionOrFail() { debugs(17, 3, HERE << entry->url()); if (serverDestinations.size() > 0) { // Ditch error page if it was created before. // A new one will be created if there's another problem delete err; err = NULL; // Update the logging information about this new server connection. // Done here before anything else so the errors get logged for // this server link regardless of what happens when connecting to it. // IF sucessfuly connected this top destination will become the serverConnection(). request->hier.note(serverDestinations[0], request->GetHost()); + request->clearError(); connectStart(); } else { debugs(17, 3, HERE << "Connection failed: " << entry->url()); if (!err) { ErrorState *anErr = new ErrorState(ERR_CANNOT_FORWARD, HTTP_INTERNAL_SERVER_ERROR, request); - anErr->xerrno = errno; fail(anErr); } // else use actual error from last connection attempt +#if USE_SSL + if (request->flags.sslPeek && request->clientConnectionManager.valid()) { + errorAppendEntry(entry, err); // will free err + err = NULL; + CallJobHere1(17, 4, request->clientConnectionManager, ConnStateData, + ConnStateData::httpsPeeked, Comm::ConnectionPointer(NULL)); + } +#endif self = NULL; // refcounted } } void FwdState::fail(ErrorState * errorState) { debugs(17, 3, HERE << err_type_str[errorState->type] << " \"" << httpStatusString(errorState->httpStatus) << "\"\n\t" << entry->url() ); delete err; err = errorState; if (!errorState->request) errorState->request = HTTPMSGLOCK(request); if (pconnRace == racePossible && err->type == ERR_ZERO_SIZE_OBJECT) { debugs(17, 5, HERE << "pconn race happened"); pconnRace = raceHappened; } - -#if USE_SSL - if (errorState->type == ERR_SECURE_CONNECT_FAIL && errorState->detail) - request->detailError(errorState->type, errorState->detail->errorNo()); - else -#endif - request->detailError(errorState->type, errorState->xerrno); } /** * Frees fwdState without closing FD or generating an abort */ void FwdState::unregister(Comm::ConnectionPointer &conn) { debugs(17, 3, HERE << entry->url() ); assert(serverConnection() == conn); assert(Comm::IsConnOpen(conn)); comm_remove_close_handler(conn->fd, fwdServerClosedWrapper, this); serverConn = NULL; } // Legacy method to be removed in favor of the above as soon as possible void FwdState::unregister(int fd) { debugs(17, 3, HERE << entry->url() ); @@ -603,182 +626,248 @@ return; case SSL_ERROR_WANT_WRITE: Comm::SetSelect(fd, COMM_SELECT_WRITE, fwdNegotiateSSLWrapper, this, 0); return; case SSL_ERROR_SSL: case SSL_ERROR_SYSCALL: ssl_lib_error = ERR_get_error(); debugs(81, 1, "fwdNegotiateSSL: Error negotiating SSL connection on FD " << fd << ": " << ERR_error_string(ssl_lib_error, NULL) << " (" << ssl_error << "/" << ret << "/" << errno << ")"); // store/report errno when ssl_error is SSL_ERROR_SYSCALL, ssl_lib_error is 0, and ret is -1 if (ssl_error == SSL_ERROR_SYSCALL && ret == -1 && ssl_lib_error == 0) sysErrNo = errno; // falling through to complete error handling default: - ErrorState *const anErr = makeConnectingError(ERR_SECURE_CONNECT_FAIL); - anErr->xerrno = sysErrNo; - + // TODO: move into a method before merge + Ssl::ErrorDetail *errDetails; Ssl::ErrorDetail *errFromFailure = (Ssl::ErrorDetail *)SSL_get_ex_data(ssl, ssl_ex_index_ssl_error_detail); if (errFromFailure != NULL) { // The errFromFailure is attached to the ssl object // and will be released when ssl object destroyed. - // Copy errFromFailure to a new Ssl::ErrorDetail object - anErr->detail = new Ssl::ErrorDetail(*errFromFailure); + // Copy errFromFailure to a new Ssl::ErrorDetail object. + errDetails = new Ssl::ErrorDetail(*errFromFailure); } else { - // clientCert can be be NULL - X509 *client_cert = SSL_get_peer_certificate(ssl); - anErr->detail = new Ssl::ErrorDetail(SQUID_ERR_SSL_HANDSHAKE, client_cert); - if (client_cert) - X509_free(client_cert); + // server_cert can be NULL here + X509 *server_cert = SSL_get_peer_certificate(ssl); + errDetails = new Ssl::ErrorDetail(SQUID_ERR_SSL_HANDSHAKE, server_cert, NULL); + X509_free(server_cert); } if (ssl_lib_error != SSL_ERROR_NONE) - anErr->detail->setLibError(ssl_lib_error); + errDetails->setLibError(ssl_lib_error); + if (request->clientConnectionManager.valid()) { + // remember the server certificate from the ErrorDetail object + if (Ssl::ServerBump *serverBump = request->clientConnectionManager->serverBump()) { + serverBump->serverCert.resetAndLock(errDetails->peerCert()); + + // remember validation errors, if any + if (Ssl::Errors *errs = static_cast(SSL_get_ex_data(ssl, ssl_ex_index_ssl_errors))) + serverBump->sslErrors = cbdataReference(errs); + } + } + + // For intercepted connections, set the host name to the server + // certificate CN. Otherwise, we just hope that CONNECT is using + // a user-entered address (a host name or a user-entered IP). + const bool isConnectRequest = !request->clientConnectionManager->port->spoof_client_ip && + !request->clientConnectionManager->port->intercepted; + if (request->flags.sslPeek && !isConnectRequest) { + if (X509 *srvX509 = errDetails->peerCert()) { + if (const char *name = Ssl::CommonHostName(srvX509)) { + request->SetHost(name); + debugs(83, 3, HERE << "reset request host: " << name); + } + } + } + + ErrorState *const anErr = makeConnectingError(ERR_SECURE_CONNECT_FAIL); + anErr->xerrno = sysErrNo; + anErr->detail = errDetails; fail(anErr); if (serverConnection()->getPeer()) { peerConnectFailed(serverConnection()->getPeer()); } serverConn->close(); return; } } + + if (request->clientConnectionManager.valid()) { + // remember the server certificate from the ErrorDetail object + if (Ssl::ServerBump *serverBump = request->clientConnectionManager->serverBump()) { + serverBump->serverCert.reset(SSL_get_peer_certificate(ssl)); + + // remember validation errors, if any + if (Ssl::Errors *errs = static_cast(SSL_get_ex_data(ssl, ssl_ex_index_ssl_errors))) + serverBump->sslErrors = cbdataReference(errs); + } + } if (serverConnection()->getPeer() && !SSL_session_reused(ssl)) { if (serverConnection()->getPeer()->sslSession) SSL_SESSION_free(serverConnection()->getPeer()->sslSession); serverConnection()->getPeer()->sslSession = SSL_get1_session(ssl); } dispatch(); } void FwdState::initiateSSL() { SSL *ssl; SSL_CTX *sslContext = NULL; const peer *peer = serverConnection()->getPeer(); int fd = serverConnection()->fd; if (peer) { assert(peer->use_ssl); sslContext = peer->sslContext; } else { sslContext = Config.ssl_client.sslContext; } assert(sslContext); if ((ssl = SSL_new(sslContext)) == NULL) { debugs(83, 1, "fwdInitiateSSL: Error allocating handle: " << ERR_error_string(ERR_get_error(), NULL) ); ErrorState *anErr = new ErrorState(ERR_SOCKET_FAILURE, HTTP_INTERNAL_SERVER_ERROR, request); - anErr->xerrno = errno; + // TODO: create Ssl::ErrorDetail with OpenSSL-supplied error code fail(anErr); self = NULL; // refcounted return; } SSL_set_fd(ssl, fd); if (peer) { if (peer->ssldomain) SSL_set_ex_data(ssl, ssl_ex_index_server, peer->ssldomain); #if NOT_YET else if (peer->name) SSL_set_ex_data(ssl, ssl_ex_index_server, peer->name); #endif else SSL_set_ex_data(ssl, ssl_ex_index_server, peer->host); if (peer->sslSession) SSL_set_session(ssl, peer->sslSession); } else { - SSL_set_ex_data(ssl, ssl_ex_index_server, (void*)request->GetHost()); - - // We need to set SNI TLS extension only in the case we are - // connecting direct to origin server - Ssl::setClientSNI(ssl, request->GetHost()); + // While we are peeking at the certificate, we may not know the server + // name that the client will request (after interception or CONNECT) + // unless it was the CONNECT request with a user-typed address. + const char *hostname = request->GetHost(); + const bool hostnameIsIp = request->GetHostIsNumeric(); + const bool isConnectRequest = !request->clientConnectionManager->port->spoof_client_ip && + !request->clientConnectionManager->port->intercepted; + if (!request->flags.sslPeek || isConnectRequest) + SSL_set_ex_data(ssl, ssl_ex_index_server, (void*)hostname); + + // Use SNI TLS extension only when we connect directly + // to the origin server and we know the server host name. + if (!hostnameIsIp) + Ssl::setClientSNI(ssl, hostname); } // Create the ACL check list now, while we have access to more info. // The list is used in ssl_verify_cb() and is freed in ssl_free(). if (acl_access *acl = Config.ssl_client.cert_error) { ACLFilledChecklist *check = new ACLFilledChecklist(acl, request, dash_str); check->fd(fd); SSL_set_ex_data(ssl, ssl_ex_index_cert_error_check, check); } + // store peeked cert to check SQUID_X509_V_ERR_CERT_CHANGE + X509 *peeked_cert; + if (request->clientConnectionManager.valid() && + request->clientConnectionManager->serverBump() && + (peeked_cert = request->clientConnectionManager->serverBump()->serverCert.get())) { + CRYPTO_add(&(peeked_cert->references),1,CRYPTO_LOCK_X509); + SSL_set_ex_data(ssl, ssl_ex_index_ssl_peeked_cert, peeked_cert); + } + fd_table[fd].ssl = ssl; fd_table[fd].read_method = &ssl_read_method; fd_table[fd].write_method = &ssl_write_method; negotiateSSL(fd); } #endif void FwdState::connectDone(const Comm::ConnectionPointer &conn, comm_err_t status, int xerrno) { if (status != COMM_OK) { ErrorState *const anErr = makeConnectingError(ERR_CONNECT_FAIL); anErr->xerrno = xerrno; fail(anErr); /* it might have been a timeout with a partially open link */ if (conn != NULL) { if (conn->getPeer()) peerConnectFailed(conn->getPeer()); conn->close(); } retryOrBail(); return; } serverConn = conn; debugs(17, 3, HERE << serverConnection() << ": '" << entry->url() << "'" ); comm_add_close_handler(serverConnection()->fd, fwdServerClosedWrapper, this); if (serverConnection()->getPeer()) peerConnectSucceded(serverConnection()->getPeer()); + // some requests benefit from pinning but do not require it and can "repin" + const bool rePin = request->flags.canRePin && + request->clientConnectionManager.valid(); + if (rePin) { + debugs(17, 3, HERE << "repinning " << serverConn); + request->clientConnectionManager->pinConnection(serverConn, + request, serverConn->getPeer(), request->flags.auth); + request->flags.pinned = 1; + } + #if USE_SSL - if (!request->flags.pinned) { + if (!request->flags.pinned || rePin) { if ((serverConnection()->getPeer() && serverConnection()->getPeer()->use_ssl) || - (!serverConnection()->getPeer() && request->protocol == AnyP::PROTO_HTTPS)) { + (!serverConnection()->getPeer() && request->protocol == AnyP::PROTO_HTTPS) || + request->flags.sslPeek) { initiateSSL(); return; } } #endif flags.connected_okay = true; dispatch(); } void FwdState::connectTimeout(int fd) { debugs(17, 2, "fwdConnectTimeout: FD " << fd << ": '" << entry->url() << "'" ); assert(serverDestinations[0] != NULL); assert(fd == serverDestinations[0]->fd); if (entry->isEmpty()) { ErrorState *anErr = new ErrorState(ERR_CONNECT_FAIL, HTTP_GATEWAY_TIMEOUT, request); anErr->xerrno = ETIMEDOUT; @@ -836,50 +925,56 @@ request->flags.pinned = 0; // XXX: what if the ConnStateData set this to flag existing credentials? // XXX: answer: the peer selection *should* catch it and give us only the pinned peer. so we reverse the =0 step below. // XXX: also, logs will now lie if pinning is broken and leads to an error message. if (serverDestinations[0]->peerType == PINNED) { ConnStateData *pinned_connection = request->pinnedConnection(); // pinned_connection may become nil after a pconn race if (pinned_connection) serverConn = pinned_connection->validatePinnedConnection(request, serverDestinations[0]->getPeer()); else serverConn = NULL; if (Comm::IsConnOpen(serverConn)) { #if 0 if (!serverConn->getPeer()) serverConn->peerType = HIER_DIRECT; #endif n_tries++; request->flags.pinned = 1; if (pinned_connection->pinnedAuth()) request->flags.auth = 1; + comm_add_close_handler(serverConn->fd, fwdServerClosedWrapper, this); // the server may close the pinned connection before this request pconnRace = racePossible; dispatch(); return; } - /* Failure. Fall back on next path */ - debugs(17,2,HERE << " Pinned connection " << pinned_connection << " not valid."); - serverDestinations.shift(); - startConnectionOrFail(); - return; + /* Failure. Fall back on next path unless we can re-pin */ + debugs(17,2,HERE << "Pinned connection failed: " << pinned_connection); + if (pconnRace != raceHappened || !request->flags.canRePin) { + serverDestinations.shift(); + pconnRace = raceImpossible; + startConnectionOrFail(); + return; + } + debugs(17,3, HERE << "There was a pconn race. Will try to repin."); + // and fall through to regular handling } // Use pconn to avoid opening a new connection. const char *host; if (serverDestinations[0]->getPeer()) { host = serverDestinations[0]->getPeer()->host; } else { host = request->GetHost(); } Comm::ConnectionPointer temp; // Avoid pconns after races so that the same client does not suffer twice. // This does not increase the total number of connections because we just // closed the connection that failed the race. And re-pinning assumes this. if (pconnRace != raceHappened) temp = fwdPconnPool->pop(serverDestinations[0], host, checkRetriable()); const bool openedPconn = Comm::IsConnOpen(temp); pconnRace = openedPconn ? racePossible : raceImpossible; @@ -969,46 +1064,57 @@ */ if (Ip::Qos::TheConfig.isHitNfmarkActive()) { if (Comm::IsConnOpen(clientConn) && Comm::IsConnOpen(serverConnection())) { fde * clientFde = &fd_table[clientConn->fd]; // XXX: move the fd_table access into Ip::Qos /* Get the netfilter mark for the connection */ Ip::Qos::getNfmarkFromServer(serverConnection(), clientFde); } } #if _SQUID_LINUX_ /* Bug 2537: The TOS forward part of QOS only applies to patched Linux kernels. */ if (Ip::Qos::TheConfig.isHitTosActive()) { if (Comm::IsConnOpen(clientConn)) { fde * clientFde = &fd_table[clientConn->fd]; // XXX: move the fd_table access into Ip::Qos /* Get the TOS value for the packet */ Ip::Qos::getTosFromServer(serverConnection(), clientFde); } } #endif +#if USE_SSL + if (request->flags.sslPeek) { + CallJobHere1(17, 4, request->clientConnectionManager, ConnStateData, + ConnStateData::httpsPeeked, serverConnection()); + unregister(serverConn); // async call owns it now + complete(); // destroys us + return; + } +#endif + if (serverConnection()->getPeer() != NULL) { serverConnection()->getPeer()->stats.fetches++; request->peer_login = serverConnection()->getPeer()->login; request->peer_domain = serverConnection()->getPeer()->domain; httpStart(this); } else { + assert(!request->flags.sslPeek); request->peer_login = NULL; request->peer_domain = NULL; switch (request->protocol) { #if USE_SSL case AnyP::PROTO_HTTPS: httpStart(this); break; #endif case AnyP::PROTO_HTTP: httpStart(this); break; case AnyP::PROTO_GOPHER: gopherStart(this); break; case AnyP::PROTO_FTP: === modified file 'src/forward.h' --- src/forward.h 2012-02-20 19:10:54 +0000 +++ src/forward.h 2012-02-24 10:13:56 +0000 @@ -52,40 +52,41 @@ void connectTimeout(int fd); void initiateSSL(); void negotiateSSL(int fd); bool checkRetry(); bool checkRetriable(); void dispatch(); void pconnPush(Comm::ConnectionPointer & conn, const char *domain); bool dontRetry() { return flags.dont_retry; } void dontRetry(bool val) { flags.dont_retry = val; } /** return a ConnectionPointer to the current server connection (may or may not be open) */ Comm::ConnectionPointer const & serverConnection() const { return serverConn; }; private: // hidden for safer management of self; use static fwdStart FwdState(const Comm::ConnectionPointer &client, StoreEntry *, HttpRequest *); void start(Pointer aSelf); + void selectPeerForIntercepted(); static void logReplyStatus(int tries, http_status status); void doneWithRetries(); void completed(); void retryOrBail(); ErrorState *makeConnectingError(const err_type type) const; static void RegisterWithCacheManager(void); public: StoreEntry *entry; HttpRequest *request; static void abort(void*); private: Pointer self; ErrorState *err; Comm::ConnectionPointer clientConn; ///< a possibly open connection to the client. time_t start_t; int n_tries; int origin_tries; === modified file 'src/ftp.cc' --- src/ftp.cc 2012-02-08 22:55:23 +0000 +++ src/ftp.cc 2012-04-18 07:22:36 +0000 @@ -3563,57 +3563,57 @@ int code = ftpState->ctrl.replycode; http_status http_code; err_type err_code = ERR_NONE; debugs(9, 3, HERE << ftpState->entry->url() << ", code " << code); if (cbdataReferenceValid(ftpState)) debugs(9, 5, HERE << "ftpState (" << ftpState << ") is valid!"); if (code == 226 || code == 250) { err_code = (ftpState->mdtm > 0) ? ERR_FTP_PUT_MODIFIED : ERR_FTP_PUT_CREATED; http_code = (ftpState->mdtm > 0) ? HTTP_ACCEPTED : HTTP_CREATED; } else if (code == 227) { err_code = ERR_FTP_PUT_CREATED; http_code = HTTP_CREATED; } else { err_code = ERR_FTP_PUT_ERROR; http_code = HTTP_INTERNAL_SERVER_ERROR; } - if (ftpState->request) - ftpState->request->detailError(err_code, code); - ErrorState err(err_code, http_code, ftpState->request); if (ftpState->old_request) err.ftp.request = xstrdup(ftpState->old_request); else err.ftp.request = xstrdup(ftpState->ctrl.last_command); if (ftpState->old_reply) err.ftp.reply = xstrdup(ftpState->old_reply); else if (ftpState->ctrl.last_reply) err.ftp.reply = xstrdup(ftpState->ctrl.last_reply); else err.ftp.reply = xstrdup(""); + // TODO: interpret as FTP-specific error code + err.detailError(code); + ftpState->entry->replaceHttpReply( err.BuildHttpReply() ); ftpSendQuit(ftpState); } void FtpStateData::appendSuccessHeader() { const char *mime_type = NULL; const char *mime_enc = NULL; String urlpath = request->urlpath; const char *filename = NULL; const char *t = NULL; debugs(9, 3, HERE); if (flags.http_header_sent) return; HttpReply *reply = new HttpReply; === modified file 'src/globals.h' --- src/globals.h 2012-04-25 05:29:20 +0000 +++ src/globals.h 2012-06-20 14:27:31 +0000 @@ -135,36 +135,38 @@ extern unsigned int WIN32_Socks_initialized; /* 0 */ #endif #if _SQUID_WINDOWS_ extern unsigned int WIN32_OS_version; /* 0 */ extern char *WIN32_OS_string; /* NULL */ extern char *WIN32_Service_name; /* NULL */ extern char *WIN32_Command_Line; /* NULL */ extern char *WIN32_Service_Command_Line; /* NULL */ extern unsigned int WIN32_run_mode; /* _WIN_SQUID_RUN_MODE_INTERACTIVE */ #endif #if HAVE_SBRK extern void *sbrk_start; /* 0 */ #endif extern int ssl_ex_index_server; /* -1 */ extern int ssl_ctx_ex_index_dont_verify_domain; /* -1 */ extern int ssl_ex_index_cert_error_check; /* -1 */ extern int ssl_ex_index_ssl_error_detail; /* -1 */ + extern int ssl_ex_index_ssl_peeked_cert; /* -1 */ + extern int ssl_ex_index_ssl_errors; /* -1 */ extern const char *external_acl_message; /* NULL */ extern int opt_send_signal; /* -1 */ extern int opt_no_daemon; /* 0 */ extern int opt_parse_cfg_only; /* 0 */ /// current Squid process number (e.g., 4). /// Zero for SMP-unaware code and in no-SMP mode. extern int KidIdentifier; /* 0 */ #ifdef __cplusplus } #endif #endif /* SQUID_GLOBALS_H */ === modified file 'src/gopher.cc' --- src/gopher.cc 2012-01-20 18:55:04 +0000 +++ src/gopher.cc 2012-04-18 07:22:36 +0000 @@ -741,85 +741,84 @@ int clen; int bin; size_t read_sz = BUFSIZ; #if USE_DELAY_POOLS DelayId delayId = entry->mem_obj->mostBytesAllowed(); #endif /* Bail out early on COMM_ERR_CLOSING - close handlers will tidy up for us */ if (flag == COMM_ERR_CLOSING) { return; } assert(buf == gopherState->replybuf); if (EBIT_TEST(entry->flags, ENTRY_ABORTED)) { gopherState->serverConn->close(); return; } - errno = 0; #if USE_DELAY_POOLS read_sz = delayId.bytesWanted(1, read_sz); #endif /* leave one space for \0 in gopherToHTML */ if (flag == COMM_OK && len > 0) { #if USE_DELAY_POOLS delayId.bytesIn(len); #endif kb_incr(&(statCounter.server.all.kbytes_in), len); kb_incr(&(statCounter.server.other.kbytes_in), len); } debugs(10, 5, HERE << conn << " read len=" << len); if (flag == COMM_OK && len > 0) { AsyncCall::Pointer nil; commSetConnTimeout(conn, Config.Timeout.read, nil); IOStats.Gopher.reads++; for (clen = len - 1, bin = 0; clen; bin++) clen >>= 1; IOStats.Gopher.read_hist[bin]++; HttpRequest *req = gopherState->fwd->request; if (req->hier.bodyBytesRead < 0) req->hier.bodyBytesRead = 0; req->hier.bodyBytesRead += len; } if (flag != COMM_OK) { debugs(50, 1, "gopherReadReply: error reading: " << xstrerror()); - if (ignoreErrno(errno)) { + if (ignoreErrno(xerrno)) { AsyncCall::Pointer call = commCbCall(5,4, "gopherReadReply", CommIoCbPtrFun(gopherReadReply, gopherState)); comm_read(conn, buf, read_sz, call); } else { ErrorState *err = new ErrorState(ERR_READ_ERROR, HTTP_INTERNAL_SERVER_ERROR, gopherState->fwd->request); - err->xerrno = errno; + err->xerrno = xerrno; gopherState->fwd->fail(err); gopherState->serverConn->close(); } } else if (len == 0 && entry->isEmpty()) { gopherState->fwd->fail(new ErrorState(ERR_ZERO_SIZE_OBJECT, HTTP_SERVICE_UNAVAILABLE, gopherState->fwd->request)); gopherState->serverConn->close(); } else if (len == 0) { /* Connection closed; retrieval done. */ /* flush the rest of data in temp buf if there is one. */ if (gopherState->conversion != gopher_ds::NORMAL) gopherEndHTML(gopherState); entry->timestampsSet(); entry->flush(); gopherState->fwd->complete(); gopherState->serverConn->close(); } else { if (gopherState->conversion != gopher_ds::NORMAL) { gopherToHTML(gopherState, buf, len); @@ -835,41 +834,41 @@ /** \ingroup ServerProtocolGopherInternal * This will be called when request write is complete. Schedule read of reply. */ static void gopherSendComplete(const Comm::ConnectionPointer &conn, char *buf, size_t size, comm_err_t errflag, int xerrno, void *data) { GopherStateData *gopherState = (GopherStateData *) data; StoreEntry *entry = gopherState->entry; debugs(10, 5, HERE << conn << " size: " << size << " errflag: " << errflag); if (size > 0) { fd_bytes(conn->fd, size, FD_WRITE); kb_incr(&(statCounter.server.all.kbytes_out), size); kb_incr(&(statCounter.server.other.kbytes_out), size); } if (errflag) { ErrorState *err; err = new ErrorState(ERR_WRITE_ERROR, HTTP_SERVICE_UNAVAILABLE, gopherState->fwd->request); - err->xerrno = errno; + err->xerrno = xerrno; err->port = gopherState->fwd->request->port; err->url = xstrdup(entry->url()); gopherState->fwd->fail(err); gopherState->serverConn->close(); if (buf) memFree(buf, MEM_4K_BUF); /* Allocated by gopherSendRequest. */ return; } /* * OK. We successfully reach remote site. Start MIME typing * stuff. Do it anyway even though request is not HTML type. */ entry->buffer(); gopherMimeCreate(gopherState); switch (gopherState->type_id) { === modified file 'src/http.cc' --- src/http.cc 2012-06-16 15:03:46 +0000 +++ src/http.cc 2012-06-11 12:52:32 +0000 @@ -2272,41 +2272,41 @@ return; } } } HttpStateData::handleMoreRequestBodyAvailable(); } // premature end of the request body void HttpStateData::handleRequestBodyProducerAborted() { ServerStateData::handleRequestBodyProducerAborted(); if (entry->isEmpty()) { debugs(11, 3, "request body aborted: " << serverConnection); // We usually get here when ICAP REQMOD aborts during body processing. // We might also get here if client-side aborts, but then our response // should not matter because either client-side will provide its own or // there will be no response at all (e.g., if the the client has left). ErrorState *err = new ErrorState(ERR_ICAP_FAILURE, HTTP_INTERNAL_SERVER_ERROR, fwd->request); - err->xerrno = ERR_DETAIL_SRV_REQMOD_REQ_BODY; + err->detailError(ERR_DETAIL_SRV_REQMOD_REQ_BODY); fwd->fail(err); } abortTransaction("request body producer aborted"); } // called when we wrote request headers(!) or a part of the body void HttpStateData::sentRequestBody(const CommIoCbParams &io) { if (io.size > 0) kb_incr(&statCounter.server.http.kbytes_out, io.size); ServerStateData::sentRequestBody(io); } // Quickly abort the transaction // TODO: destruction should be sufficient as the destructor should cleanup, // including canceling close handlers void === modified file 'src/ssl/ErrorDetail.cc' --- src/ssl/ErrorDetail.cc 2012-01-20 18:55:04 +0000 +++ src/ssl/ErrorDetail.cc 2012-06-20 14:27:31 +0000 @@ -1,38 +1,40 @@ #include "squid-old.h" #include "errorpage.h" #include "ssl/ErrorDetail.h" #if HAVE_MAP #include #endif struct SslErrorEntry { Ssl::ssl_error_t value; const char *name; }; static const char *SslErrorDetailDefaultStr = "SSL handshake error (%err_name)"; //Use std::map to optimize search typedef std::map SslErrors; SslErrors TheSslErrors; static SslErrorEntry TheSslErrorArray[] = { + {SQUID_X509_V_ERR_CERT_CHANGE, + "SQUID_X509_V_ERR_CERT_CHANGE"}, {SQUID_ERR_SSL_HANDSHAKE, "SQUID_ERR_SSL_HANDSHAKE"}, {SQUID_X509_V_ERR_DOMAIN_MISMATCH, "SQUID_X509_V_ERR_DOMAIN_MISMATCH"}, {X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT, "X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT"}, {X509_V_ERR_UNABLE_TO_GET_CRL, "X509_V_ERR_UNABLE_TO_GET_CRL"}, {X509_V_ERR_UNABLE_TO_DECRYPT_CERT_SIGNATURE, "X509_V_ERR_UNABLE_TO_DECRYPT_CERT_SIGNATURE"}, {X509_V_ERR_UNABLE_TO_DECRYPT_CRL_SIGNATURE, "X509_V_ERR_UNABLE_TO_DECRYPT_CRL_SIGNATURE"}, {X509_V_ERR_UNABLE_TO_DECODE_ISSUER_PUBLIC_KEY, "X509_V_ERR_UNABLE_TO_DECODE_ISSUER_PUBLIC_KEY"}, {X509_V_ERR_CERT_SIGNATURE_FAILURE, "X509_V_ERR_CERT_SIGNATURE_FAILURE"}, {X509_V_ERR_CRL_SIGNATURE_FAILURE, "X509_V_ERR_CRL_SIGNATURE_FAILURE"}, {X509_V_ERR_CERT_NOT_YET_VALID, "X509_V_ERR_CERT_NOT_YET_VALID"}, @@ -71,184 +73,243 @@ {X509_V_ERR_INVALID_PURPOSE, "X509_V_ERR_INVALID_PURPOSE"}, {X509_V_ERR_CERT_UNTRUSTED, "X509_V_ERR_CERT_UNTRUSTED"}, {X509_V_ERR_CERT_REJECTED, "X509_V_ERR_CERT_REJECTED"}, {X509_V_ERR_SUBJECT_ISSUER_MISMATCH, "X509_V_ERR_SUBJECT_ISSUER_MISMATCH"}, {X509_V_ERR_AKID_SKID_MISMATCH, "X509_V_ERR_AKID_SKID_MISMATCH"}, {X509_V_ERR_AKID_ISSUER_SERIAL_MISMATCH, "X509_V_ERR_AKID_ISSUER_SERIAL_MISMATCH"}, {X509_V_ERR_KEYUSAGE_NO_CERTSIGN, "X509_V_ERR_KEYUSAGE_NO_CERTSIGN"}, {X509_V_ERR_APPLICATION_VERIFICATION, "X509_V_ERR_APPLICATION_VERIFICATION"}, { SSL_ERROR_NONE, "SSL_ERROR_NONE"}, {SSL_ERROR_NONE, NULL} }; +struct SslErrorAlias { + const char *name; + const Ssl::ssl_error_t *errors; +}; + +static const Ssl::ssl_error_t hasExpired[] = {X509_V_ERR_CERT_HAS_EXPIRED, SSL_ERROR_NONE}; +static const Ssl::ssl_error_t notYetValid[] = {X509_V_ERR_CERT_NOT_YET_VALID, SSL_ERROR_NONE}; +static const Ssl::ssl_error_t domainMismatch[] = {SQUID_X509_V_ERR_DOMAIN_MISMATCH, SSL_ERROR_NONE}; +static const Ssl::ssl_error_t certUntrusted[] = {X509_V_ERR_INVALID_CA, + X509_V_ERR_SELF_SIGNED_CERT_IN_CHAIN, + X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE, + X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT, + X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY, + X509_V_ERR_CERT_UNTRUSTED, SSL_ERROR_NONE}; +static const Ssl::ssl_error_t certSelfSigned[] = {X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT, SSL_ERROR_NONE}; + +// The list of error name shortcuts for use with ssl_error acls. +// The keys without the "ssl::" scope prefix allow shorter error +// names within the SSL options scope. This is easier than +// carefully stripping the scope prefix in Ssl::ParseErrorString(). +static SslErrorAlias TheSslErrorShortcutsArray[] = { + {"ssl::certHasExpired", hasExpired}, + {"certHasExpired", hasExpired}, + {"ssl::certNotYetValid", notYetValid}, + {"certNotYetValid", notYetValid}, + {"ssl::certDomainMismatch", domainMismatch}, + {"certDomainMismatch", domainMismatch}, + {"ssl::certUntrusted", certUntrusted}, + {"certUntrusted", certUntrusted}, + {"ssl::certSelfSigned", certSelfSigned}, + {"certSelfSigned", certSelfSigned}, + {NULL, NULL} +}; + +// Use std::map to optimize search. +typedef std::map SslErrorShortcuts; +SslErrorShortcuts TheSslErrorShortcuts; + static void loadSslErrorMap() { assert(TheSslErrors.empty()); for (int i = 0; TheSslErrorArray[i].name; ++i) { TheSslErrors[TheSslErrorArray[i].value] = &TheSslErrorArray[i]; } } +static void loadSslErrorShortcutsMap() +{ + assert(TheSslErrorShortcuts.empty()); + for (int i = 0; TheSslErrorShortcutsArray[i].name; i++) + TheSslErrorShortcuts[TheSslErrorShortcutsArray[i].name] = TheSslErrorShortcutsArray[i].errors; +} + Ssl::ssl_error_t Ssl::GetErrorCode(const char *name) { for (int i = 0; TheSslErrorArray[i].name != NULL; i++) { if (strcmp(name, TheSslErrorArray[i].name) == 0) return TheSslErrorArray[i].value; } return SSL_ERROR_NONE; } -Ssl::ssl_error_t +Ssl::Errors * Ssl::ParseErrorString(const char *name) { assert(name); const Ssl::ssl_error_t ssl_error = GetErrorCode(name); if (ssl_error != SSL_ERROR_NONE) - return ssl_error; + return new Ssl::Errors(ssl_error); if (xisdigit(*name)) { const long int value = strtol(name, NULL, 0); if (SQUID_SSL_ERROR_MIN <= value && value <= SQUID_SSL_ERROR_MAX) - return value; + return new Ssl::Errors(value); fatalf("Too small or too bug SSL error code '%s'", name); } + if (TheSslErrorShortcuts.empty()) + loadSslErrorShortcutsMap(); + + const SslErrorShortcuts::const_iterator it = TheSslErrorShortcuts.find(name); + if (it != TheSslErrorShortcuts.end()) { + // Should not be empty... + assert(it->second[0] != SSL_ERROR_NONE); + Ssl::Errors *errors = new Ssl::Errors(it->second[0]); + for (int i =1; it->second[i] != SSL_ERROR_NONE; i++) { + errors->push_back_unique(it->second[i]); + } + return errors; + } + fatalf("Unknown SSL error name '%s'", name); - return SSL_ERROR_SSL; // not reached + return NULL; // not reached } const char *Ssl::GetErrorName(Ssl::ssl_error_t value) { if (TheSslErrors.empty()) loadSslErrorMap(); const SslErrors::const_iterator it = TheSslErrors.find(value); if (it != TheSslErrors.end()) return it->second->name; return NULL; } const char * Ssl::GetErrorDescr(Ssl::ssl_error_t value) { return ErrorDetailsManager::GetInstance().getDefaultErrorDescr(value); } Ssl::ErrorDetail::err_frm_code Ssl::ErrorDetail::ErrorFormatingCodes[] = { {"ssl_subject", &Ssl::ErrorDetail::subject}, {"ssl_ca_name", &Ssl::ErrorDetail::ca_name}, {"ssl_cn", &Ssl::ErrorDetail::cn}, {"ssl_notbefore", &Ssl::ErrorDetail::notbefore}, {"ssl_notafter", &Ssl::ErrorDetail::notafter}, {"err_name", &Ssl::ErrorDetail::err_code}, {"ssl_error_descr", &Ssl::ErrorDetail::err_descr}, {"ssl_lib_error", &Ssl::ErrorDetail::err_lib_error}, {NULL,NULL} }; /** * The subject of the current certification in text form */ const char *Ssl::ErrorDetail::subject() const { - if (!peer_cert) + if (!broken_cert) return "[Not available]"; static char tmpBuffer[256]; // A temporary buffer - X509_NAME_oneline(X509_get_subject_name(peer_cert.get()), tmpBuffer, + X509_NAME_oneline(X509_get_subject_name(broken_cert.get()), tmpBuffer, sizeof(tmpBuffer)); return tmpBuffer; } // helper function to be used with Ssl::matchX509CommonNames static int copy_cn(void *check_data, ASN1_STRING *cn_data) { String *str = (String *)check_data; if (!str) // no data? abort return 0; if (str->defined()) str->append(", "); str->append((const char *)cn_data->data, cn_data->length); return 1; } /** * The list with certificates cn and alternate names */ const char *Ssl::ErrorDetail::cn() const { - if (!peer_cert) + if (!broken_cert) return "[Not available]"; static String tmpStr; ///< A temporary string buffer tmpStr.clean(); - Ssl::matchX509CommonNames(peer_cert.get(), &tmpStr, copy_cn); + Ssl::matchX509CommonNames(broken_cert.get(), &tmpStr, copy_cn); return tmpStr.termedBuf(); } /** * The issuer name */ const char *Ssl::ErrorDetail::ca_name() const { - if (!peer_cert) + if (!broken_cert) return "[Not available]"; static char tmpBuffer[256]; // A temporary buffer - X509_NAME_oneline(X509_get_issuer_name(peer_cert.get()), tmpBuffer, sizeof(tmpBuffer)); + X509_NAME_oneline(X509_get_issuer_name(broken_cert.get()), tmpBuffer, sizeof(tmpBuffer)); return tmpBuffer; } /** * The certificate "not before" field */ const char *Ssl::ErrorDetail::notbefore() const { - if (!peer_cert) + if (!broken_cert) return "[Not available]"; static char tmpBuffer[256]; // A temporary buffer - ASN1_UTCTIME * tm = X509_get_notBefore(peer_cert.get()); + ASN1_UTCTIME * tm = X509_get_notBefore(broken_cert.get()); Ssl::asn1timeToString(tm, tmpBuffer, sizeof(tmpBuffer)); return tmpBuffer; } /** * The certificate "not after" field */ const char *Ssl::ErrorDetail::notafter() const { - if (!peer_cert) + if (!broken_cert) return "[Not available]"; static char tmpBuffer[256]; // A temporary buffer - ASN1_UTCTIME * tm = X509_get_notAfter(peer_cert.get()); + ASN1_UTCTIME * tm = X509_get_notAfter(broken_cert.get()); Ssl::asn1timeToString(tm, tmpBuffer, sizeof(tmpBuffer)); return tmpBuffer; } /** * The string representation of the error_no */ const char *Ssl::ErrorDetail::err_code() const { static char tmpBuffer[64]; // We can use the GetErrorName but using the detailEntry is faster, // so try it first. const char *err = detailEntry.name.termedBuf(); // error details not loaded yet or not defined in error_details.txt, // try the GetErrorName... if (!err) err = GetErrorName(error_no); if (!err) { @@ -262,50 +323,54 @@ * A short description of the error_no */ const char *Ssl::ErrorDetail::err_descr() const { if (error_no == SSL_ERROR_NONE) return "[No Error]"; if (const char *err = detailEntry.descr.termedBuf()) return err; return "[Not available]"; } const char *Ssl::ErrorDetail::err_lib_error() const { if (lib_error_no != SSL_ERROR_NONE) return ERR_error_string(lib_error_no, NULL); else return "[No Error]"; } /** - * It converts the code to a string value. Currently the following - * formating codes are supported: + * Converts the code to a string value. Supported formating codes are: + * + * Error meta information: * %err_name: The name of a high-level SSL error (e.g., X509_V_ERR_*) * %ssl_error_descr: A short description of the SSL error + * %ssl_lib_error: human-readable low-level error string by ERR_error_string(3SSL) + * + * Certificate information extracted from broken (not necessarily peer!) cert * %ssl_cn: The comma-separated list of common and alternate names * %ssl_subject: The certificate subject * %ssl_ca_name: The certificate issuer name * %ssl_notbefore: The certificate "not before" field * %ssl_notafter: The certificate "not after" field - * %ssl_lib_error: human-readable low-level error string by ERR_error_string(3SSL) + * \retval the length of the code (the number of characters will be replaced by value) */ int Ssl::ErrorDetail::convert(const char *code, const char **value) const { *value = "-"; for (int i=0; ErrorFormatingCodes[i].code!=NULL; i++) { const int len = strlen(ErrorFormatingCodes[i].code); if (strncmp(code,ErrorFormatingCodes[i].code, len)==0) { ErrorDetail::fmt_action_t action = ErrorFormatingCodes[i].fmt_action; *value = (this->*action)(); return len; } } return 0; } /** * It uses the convert method to build the string errDetailStr using * a template message for the current SSL error. The template messages * can also contain normal error pages formating codes. @@ -327,46 +392,50 @@ assert(s); while ((p = strchr(s, '%'))) { errDetailStr.append(s, p - s); code_len = convert(++p, &t); if (code_len) errDetailStr.append(t); else errDetailStr.append("%"); s = p + code_len; } errDetailStr.append(s, strlen(s)); } const String &Ssl::ErrorDetail::toString() const { if (!errDetailStr.defined()) buildDetail(); return errDetailStr; } -/* We may do not want to use X509_dup but instead - internal SSL locking: - CRYPTO_add(&(cert->references),1,CRYPTO_LOCK_X509); - peer_cert.reset(cert); -*/ -Ssl::ErrorDetail::ErrorDetail( Ssl::ssl_error_t err_no, X509 *cert): error_no (err_no), lib_error_no(SSL_ERROR_NONE) +Ssl::ErrorDetail::ErrorDetail( Ssl::ssl_error_t err_no, X509 *cert, X509 *broken): error_no (err_no), lib_error_no(SSL_ERROR_NONE) { if (cert) - peer_cert.reset(X509_dup(cert)); + peer_cert.resetAndLock(cert); + + if (broken) + broken_cert.resetAndLock(broken); + else + broken_cert.resetAndLock(cert); detailEntry.error_no = SSL_ERROR_NONE; } Ssl::ErrorDetail::ErrorDetail(Ssl::ErrorDetail const &anErrDetail) { error_no = anErrDetail.error_no; request = anErrDetail.request; if (anErrDetail.peer_cert.get()) { - peer_cert.reset(X509_dup(anErrDetail.peer_cert.get())); + peer_cert.resetAndLock(anErrDetail.peer_cert.get()); + } + + if (anErrDetail.broken_cert.get()) { + broken_cert.resetAndLock(anErrDetail.broken_cert.get()); } detailEntry = anErrDetail.detailEntry; lib_error_no = anErrDetail.lib_error_no; } === modified file 'src/ssl/ErrorDetail.h' --- src/ssl/ErrorDetail.h 2011-11-18 16:40:10 +0000 +++ src/ssl/ErrorDetail.h 2012-06-20 14:27:31 +0000 @@ -1,94 +1,101 @@ #ifndef _SQUID_SSL_ERROR_DETAIL_H #define _SQUID_SSL_ERROR_DETAIL_H #include "err_detail_type.h" #include "HttpRequest.h" #include "ErrorDetailManager.h" #include "ssl/support.h" #include "ssl/gadgets.h" #if HAVE_OPENSSL_SSL_H #include #endif namespace Ssl { /** \ingroup ServerProtocolSSLAPI - * The ssl_error_t representation of the error described by "name". - * This function also parses numeric arguments. + * Converts user-friendly error "name" into an Ssl::Errors list. + * The resulting list may have one or more elements, and needs to be + * released by the caller. + * This function can handle numeric error numbers as well as names. */ -ssl_error_t ParseErrorString(const char *name); +Ssl::Errors *ParseErrorString(const char *name); /** \ingroup ServerProtocolSSLAPI * The ssl_error_t code of the error described by "name". */ ssl_error_t GetErrorCode(const char *name); /** \ingroup ServerProtocolSSLAPI * The string representation of the SSL error "value" */ const char *GetErrorName(ssl_error_t value); /** \ingroup ServerProtocolSSLAPI * A short description of the SSL error "value" */ const char *GetErrorDescr(ssl_error_t value); /** \ingroup ServerProtocolSSLAPI * Used to pass SSL error details to the error pages returned to the * end user. */ class ErrorDetail { public: - ErrorDetail(ssl_error_t err_no, X509 *cert); + // if broken certificate is nil, the peer certificate is broken + ErrorDetail(ssl_error_t err_no, X509 *peer, X509 *broken); ErrorDetail(ErrorDetail const &); const String &toString() const; ///< An error detail string to embed in squid error pages void useRequest(HttpRequest *aRequest) { if (aRequest != NULL) request = aRequest;} /// The error name to embed in squid error pages const char *errorName() const {return err_code();} /// The error no ssl_error_t errorNo() const {return error_no;} ///Sets the low-level error returned by OpenSSL ERR_get_error() void setLibError(unsigned long lib_err_no) {lib_error_no = lib_err_no;} - + /// the peer certificate + X509 *peerCert() { return peer_cert.get(); } + /// peer or intermediate certificate that failed validation + X509 *brokenCert() {return broken_cert.get(); } private: typedef const char * (ErrorDetail::*fmt_action_t)() const; /** * Holds a formating code and its conversion method */ class err_frm_code { public: const char *code; ///< The formating code fmt_action_t fmt_action; ///< A pointer to the conversion method }; static err_frm_code ErrorFormatingCodes[]; ///< The supported formating codes const char *subject() const; const char *ca_name() const; const char *cn() const; const char *notbefore() const; const char *notafter() const; const char *err_code() const; const char *err_descr() const; const char *err_lib_error() const; int convert(const char *code, const char **value) const; void buildDetail() const; mutable String errDetailStr; ///< Caches the error detail message ssl_error_t error_no; ///< The error code unsigned long lib_error_no; ///< low-level error returned by OpenSSL ERR_get_error(3SSL) X509_Pointer peer_cert; ///< A pointer to the peer certificate + X509_Pointer broken_cert; ///< A pointer to the broken certificate (peer or intermediate) mutable ErrorDetailEntry detailEntry; HttpRequest::Pointer request; }; }//namespace Ssl #endif === modified file 'src/ssl/Makefile.am' --- src/ssl/Makefile.am 2012-06-08 11:33:21 +0000 +++ src/ssl/Makefile.am 2012-06-20 14:27:31 +0000 @@ -14,39 +14,41 @@ if USE_SSL_CRTD SSL_CRTD = ssl_crtd SSL_CRTD_SOURCE = \ helper.cc \ helper.h else SSL_CRTD = SSL_CRTD_SOURCE = endif ## SSL stuff used by main Squid but not by ssl_crtd libsslsquid_la_SOURCES = \ context_storage.cc \ context_storage.h \ Config.cc \ Config.h \ ErrorDetail.cc \ ErrorDetail.h \ ErrorDetailManager.cc \ ErrorDetailManager.h \ + ServerBump.cc \ + ServerBump.h \ support.cc \ support.h \ \ $(SSL_CRTD_SOURCE) ## SSL stuff used by main Squid and ssl_crtd libsslutil_la_SOURCES = \ gadgets.cc \ gadgets.h \ crtd_message.cc \ crtd_message.h libexec_PROGRAMS = \ $(SSL_CRTD) if USE_SSL_CRTD ssl_crtd_SOURCES = ssl_crtd.cc certificate_db.cc certificate_db.h ssl_crtd_LDADD = libsslutil.la $(SSLLIB) $(COMPAT_LIB) endif === added file 'src/ssl/ServerBump.cc' --- src/ssl/ServerBump.cc 1970-01-01 00:00:00 +0000 +++ src/ssl/ServerBump.cc 2012-06-20 14:27:31 +0000 @@ -0,0 +1,45 @@ +/* + * $Id$ + * + * DEBUG: section 33 Client-side Routines + * + */ + +#include "squid.h" + +#include "client_side.h" +#include "forward.h" +#include "ssl/ServerBump.h" +#include "Store.h" + + +CBDATA_NAMESPACED_CLASS_INIT(Ssl, ServerBump); + + +Ssl::ServerBump::ServerBump(HttpRequest *fakeRequest, StoreEntry *e): + request(fakeRequest), + sslErrors(NULL) +{ + debugs(33, 4, HERE << "will peek at " << request->GetHost() << ':' << request->port); + const char *uri = urlCanonical(request); + if (e) { + entry = e; + entry->lock(); + } else + entry = storeCreateEntry(uri, uri, request->flags, request->method); + // We do not need to be a client because the error contents will be used + // later, but an entry without any client will trim all its contents away. + sc = storeClientListAdd(entry, this); +} + +Ssl::ServerBump::~ServerBump() +{ + debugs(33, 4, HERE << "destroying"); + if (entry) { + debugs(33, 4, HERE << *entry); + storeUnregister(sc, entry, this); + entry->unlock(); + } + cbdataReferenceDone(sslErrors); +} + === added file 'src/ssl/ServerBump.h' --- src/ssl/ServerBump.h 1970-01-01 00:00:00 +0000 +++ src/ssl/ServerBump.h 2012-06-20 14:27:31 +0000 @@ -0,0 +1,39 @@ +#ifndef _SQUID_SSL_PEEKER_H +#define _SQUID_SSL_PEEKER_H + +#include "base/AsyncJob.h" +#include "base/CbcPointer.h" +#include "comm/forward.h" +#include "HttpRequest.h" +#include "ip/Address.h" + +class ConnStateData; + +namespace Ssl +{ + +/** + \ingroup ServerProtocolSSLAPI + * Maintains bump-server-first related information. + */ +class ServerBump +{ +public: + explicit ServerBump(HttpRequest *fakeRequest, StoreEntry *e = NULL); + ~ServerBump(); + + /// faked, minimal request; required by server-side API + HttpRequest::Pointer request; + StoreEntry *entry; ///< for receiving Squid-generated error messages + Ssl::X509_Pointer serverCert; ///< HTTPS server certificate + Ssl::Errors *sslErrors; ///< SSL [certificate validation] errors + +private: + store_client *sc; ///< dummy client to prevent entry trimming + + CBDATA_CLASS2(ServerBump); +}; + +} // namespace Ssl + +#endif === modified file 'src/ssl/certificate_db.cc' --- src/ssl/certificate_db.cc 2012-06-16 02:17:14 +0000 +++ src/ssl/certificate_db.cc 2012-06-20 14:27:31 +0000 @@ -154,238 +154,184 @@ } int Ssl::CertificateDb::index_serial_cmp(const char **a, const char **b) { const char *aa, *bb; for (aa = a[Ssl::CertificateDb::cnlSerial]; *aa == '0'; aa++); for (bb = b[Ssl::CertificateDb::cnlSerial]; *bb == '0'; bb++); return strcmp(aa, bb); } unsigned long Ssl::CertificateDb::index_name_hash(const char **a) { return(lh_strhash(a[Ssl::CertificateDb::cnlName])); } int Ssl::CertificateDb::index_name_cmp(const char **a, const char **b) { return(strcmp(a[Ssl::CertificateDb::cnlName], b[CertificateDb::cnlName])); } -const std::string Ssl::CertificateDb::serial_file("serial"); const std::string Ssl::CertificateDb::db_file("index.txt"); const std::string Ssl::CertificateDb::cert_dir("certs"); const std::string Ssl::CertificateDb::size_file("size"); Ssl::CertificateDb::CertificateDb(std::string const & aDb_path, size_t aMax_db_size, size_t aFs_block_size) : db_path(aDb_path), - serial_full(aDb_path + "/" + serial_file), db_full(aDb_path + "/" + db_file), cert_full(aDb_path + "/" + cert_dir), size_full(aDb_path + "/" + size_file), db(NULL), max_db_size(aMax_db_size), fs_block_size(aFs_block_size), dbLock(db_full), - dbSerialLock(serial_full), enabled_disk_store(true) { if (db_path.empty() && !max_db_size) enabled_disk_store = false; else if ((db_path.empty() && max_db_size) || (!db_path.empty() && !max_db_size)) throw std::runtime_error("ssl_crtd is missing the required parameter. There should be -s and -M parameters together."); } bool Ssl::CertificateDb::find(std::string const & host_name, Ssl::X509_Pointer & cert, Ssl::EVP_PKEY_Pointer & pkey) { const Locker locker(dbLock, Here); load(); return pure_find(host_name, cert, pkey); } -bool Ssl::CertificateDb::addCertAndPrivateKey(Ssl::X509_Pointer & cert, Ssl::EVP_PKEY_Pointer & pkey) +bool Ssl::CertificateDb::purgeCert(std::string const & key) +{ + const Locker locker(dbLock, Here); + load(); + if (!db) + return false; + + if (!deleteByHostname(key)) + return false; + + save(); + return true; +} + +bool Ssl::CertificateDb::addCertAndPrivateKey(Ssl::X509_Pointer & cert, Ssl::EVP_PKEY_Pointer & pkey, std::string const & useName) { const Locker locker(dbLock, Here); load(); if (!db || !cert || !pkey) return false; Row row; ASN1_INTEGER * ai = X509_get_serialNumber(cert.get()); std::string serial_string; Ssl::BIGNUM_Pointer serial(ASN1_INTEGER_to_BN(ai, NULL)); { TidyPointer hex_bn(BN_bn2hex(serial.get())); serial_string = std::string(hex_bn.get()); } row.setValue(cnlSerial, serial_string.c_str()); char ** rrow = TXT_DB_get_by_index(db.get(), cnlSerial, row.getRow()); + // We are creating certificates with unique serial numbers. If the serial + // number is found in the database, the same certificate is already stored. if (rrow != NULL) - return false; + return true; { TidyPointer subject(X509_NAME_oneline(X509_get_subject_name(cert.get()), NULL, 0)); - if (pure_find(subject.get(), cert, pkey)) + if (pure_find(useName.empty() ? subject.get() : useName, cert, pkey)) return true; } // check db size while trying to minimize calls to size() while (size() > max_db_size) { if (deleteInvalidCertificate()) continue; // try to find another invalid certificate if needed // there are no more invalid ones, but there must be valid certificates do { if (!deleteOldestCertificate()) return false; // errors prevented us from freeing enough space } while (size() > max_db_size); break; } row.setValue(cnlType, "V"); ASN1_UTCTIME * tm = X509_get_notAfter(cert.get()); row.setValue(cnlExp_date, std::string(reinterpret_cast(tm->data), tm->length).c_str()); row.setValue(cnlFile, "unknown"); - { + if (!useName.empty()) + row.setValue(cnlName, useName.c_str()); + else { TidyPointer subject(X509_NAME_oneline(X509_get_subject_name(cert.get()), NULL, 0)); row.setValue(cnlName, subject.get()); } if (!TXT_DB_insert(db.get(), row.getRow())) return false; row.reset(); std::string filename(cert_full + "/" + serial_string + ".pem"); if (!writeCertAndPrivateKeyToFile(cert, pkey, filename.c_str())) return false; addSize(filename); save(); return true; } -BIGNUM * Ssl::CertificateDb::getCurrentSerialNumber() -{ - const Locker locker(dbSerialLock, Here); - // load serial number from file. - Ssl::BIO_Pointer file(BIO_new(BIO_s_file())); - if (!file) - return NULL; - - if (BIO_rw_filename(file.get(), const_cast(serial_full.c_str())) <= 0) - return NULL; - - Ssl::ASN1_INT_Pointer serial_ai(ASN1_INTEGER_new()); - if (!serial_ai) - return NULL; - - char buffer[1024]; - if (!a2i_ASN1_INTEGER(file.get(), serial_ai.get(), buffer, sizeof(buffer))) - return NULL; - - Ssl::BIGNUM_Pointer serial(ASN1_INTEGER_to_BN(serial_ai.get(), NULL)); - - if (!serial) - return NULL; - - // increase serial number. - Ssl::BIGNUM_Pointer increased_serial(BN_dup(serial.get())); - if (!increased_serial) - return NULL; - - BN_add_word(increased_serial.get(), 1); - - // save increased serial number. - if (BIO_seek(file.get(), 0)) - return NULL; - - Ssl::ASN1_INT_Pointer increased_serial_ai(BN_to_ASN1_INTEGER(increased_serial.get(), NULL)); - if (!increased_serial_ai) - return NULL; - - i2a_ASN1_INTEGER(file.get(), increased_serial_ai.get()); - BIO_puts(file.get(),"\n"); - - return serial.release(); -} - -void Ssl::CertificateDb::create(std::string const & db_path, int serial) +void Ssl::CertificateDb::create(std::string const & db_path) { if (db_path == "") throw std::runtime_error("Path to db is empty"); - std::string serial_full(db_path + "/" + serial_file); std::string db_full(db_path + "/" + db_file); std::string cert_full(db_path + "/" + cert_dir); std::string size_full(db_path + "/" + size_file); #if _SQUID_MSWIN_ if (mkdir(db_path.c_str())) #else if (mkdir(db_path.c_str(), 0777)) #endif throw std::runtime_error("Cannot create " + db_path); #if _SQUID_MSWIN_ if (mkdir(cert_full.c_str())) #else if (mkdir(cert_full.c_str(), 0777)) #endif throw std::runtime_error("Cannot create " + cert_full); - Ssl::ASN1_INT_Pointer i(ASN1_INTEGER_new()); - ASN1_INTEGER_set(i.get(), serial); - - Ssl::BIO_Pointer file(BIO_new(BIO_s_file())); - if (!file) - throw std::runtime_error("SSL error"); - - if (BIO_write_filename(file.get(), const_cast(serial_full.c_str())) <= 0) - throw std::runtime_error("Cannot open " + cert_full + " to open"); - - i2a_ASN1_INTEGER(file.get(), i.get()); - std::ofstream size(size_full.c_str()); if (size) size << 0; else throw std::runtime_error("Cannot open " + size_full + " to open"); std::ofstream db(db_full.c_str()); if (!db) throw std::runtime_error("Cannot open " + db_full + " to open"); } void Ssl::CertificateDb::check(std::string const & db_path, size_t max_db_size) { CertificateDb db(db_path, max_db_size, 0); db.load(); } -std::string Ssl::CertificateDb::getSNString() const -{ - const Locker locker(dbSerialLock, Here); - std::ifstream file(serial_full.c_str()); - if (!file) - return ""; - std::string serial; - file >> serial; - return serial; -} - bool Ssl::CertificateDb::pure_find(std::string const & host_name, Ssl::X509_Pointer & cert, Ssl::EVP_PKEY_Pointer & pkey) { if (!db) return false; Row row; row.setValue(cnlName, host_name.c_str()); char **rrow = TXT_DB_get_by_index(db.get(), cnlName, row.getRow()); if (rrow == NULL) return false; if (!sslDateIsInTheFuture(rrow[cnlExp_date])) { deleteByHostname(rrow[cnlName]); return false; } // read cert and pkey from file. std::string filename(cert_full + "/" + rrow[cnlSerial] + ".pem"); readCertAndPrivateKeyFromFiles(cert, pkey, filename.c_str(), NULL); === modified file 'src/ssl/certificate_db.h' --- src/ssl/certificate_db.h 2012-03-08 13:03:19 +0000 +++ src/ssl/certificate_db.h 2012-03-27 16:29:42 +0000 @@ -77,49 +77,48 @@ /// A wrapper for OpenSSL database row of TXT_DB database. class Row { public: /// Create row wrapper. Row(); /// Delete all row. ~Row(); void setValue(size_t number, char const * value); ///< Set cell's value in row char ** getRow(); ///< Raw row void reset(); ///< Abandon row and don't free memory private: char **row; ///< Raw row size_t width; ///< Number of cells in the row }; CertificateDb(std::string const & db_path, size_t aMax_db_size, size_t aFs_block_size); /// Find certificate and private key for host name bool find(std::string const & host_name, Ssl::X509_Pointer & cert, Ssl::EVP_PKEY_Pointer & pkey); + /// Delete a certificate from database + bool purgeCert(std::string const & key); /// Save certificate to disk. - bool addCertAndPrivateKey(Ssl::X509_Pointer & cert, Ssl::EVP_PKEY_Pointer & pkey); - /// Get a serial number to use for generating a new certificate. - BIGNUM * getCurrentSerialNumber(); + bool addCertAndPrivateKey(Ssl::X509_Pointer & cert, Ssl::EVP_PKEY_Pointer & pkey, std::string const & useName); /// Create and initialize a database under the db_path - static void create(std::string const & db_path, int serial); + static void create(std::string const & db_path); /// Check the database stored under the db_path. static void check(std::string const & db_path, size_t max_db_size); - std::string getSNString() const; ///< Get serial number as string. bool IsEnabledDiskStore() const; ///< Check enabled of dist store. private: void load(); ///< Load db from disk. void save(); ///< Save db to disk. size_t size() const; ///< Get db size on disk in bytes. /// Increase db size by the given file size and update size_file void addSize(std::string const & filename); /// Decrease db size by the given file size and update size_file void subSize(std::string const & filename); size_t readSize() const; ///< Read size from file size_file void writeSize(size_t db_size); ///< Write size to file size_file. size_t getFileSize(std::string const & filename); ///< get file size on disk. /// Only find certificate in current db and return it. bool pure_find(std::string const & host_name, Ssl::X509_Pointer & cert, Ssl::EVP_PKEY_Pointer & pkey); void deleteRow(const char **row, int rowIndex); ///< Delete a row from TXT_DB bool deleteInvalidCertificate(); ///< Delete invalid certificate. bool deleteOldestCertificate(); ///< Delete oldest certificate. bool deleteByHostname(std::string const & host); ///< Delete using host name. @@ -137,44 +136,41 @@ #if OPENSSL_VERSION_NUMBER > 0x10000000L static unsigned long index_serial_LHASH_HASH(const void *a) { return index_serial_hash((const char **)a); } static int index_serial_LHASH_COMP(const void *arg1, const void *arg2) { return index_serial_cmp((const char **)arg1, (const char **)arg2); } static unsigned long index_name_LHASH_HASH(const void *a) { return index_name_hash((const char **)a); } static int index_name_LHASH_COMP(const void *arg1, const void *arg2) { return index_name_cmp((const char **)arg1, (const char **)arg2); } #else static IMPLEMENT_LHASH_HASH_FN(index_serial_hash,const char **) static IMPLEMENT_LHASH_COMP_FN(index_serial_cmp,const char **) static IMPLEMENT_LHASH_HASH_FN(index_name_hash,const char **) static IMPLEMENT_LHASH_COMP_FN(index_name_cmp,const char **) #endif - static const std::string serial_file; ///< Base name of the file to store serial number. static const std::string db_file; ///< Base name of the database index file. static const std::string cert_dir; ///< Base name of the directory to store the certs. static const std::string size_file; ///< Base name of the file to store db size. /// Min size of disk db. If real size < min_db_size the db will be disabled. static const size_t min_db_size; const std::string db_path; ///< The database directory. - const std::string serial_full; ///< Full path of the file to store serial number. const std::string db_full; ///< Full path of the database index file. const std::string cert_full; ///< Full path of the directory to store the certs. const std::string size_full; ///< Full path of the file to store the db size. TXT_DB_Pointer db; ///< Database with certificates info. const size_t max_db_size; ///< Max size of db. const size_t fs_block_size; ///< File system block size. mutable Lock dbLock; ///< protects the database file - mutable Lock dbSerialLock; ///< protects the serial number file bool enabled_disk_store; ///< The storage on the disk is enabled. }; } // namespace Ssl #endif // SQUID_SSL_CERTIFICATE_DB_H === modified file 'src/ssl/crtd_message.cc' --- src/ssl/crtd_message.cc 2012-01-20 18:55:04 +0000 +++ src/ssl/crtd_message.cc 2012-03-08 12:41:28 +0000 @@ -1,32 +1,36 @@ /* * $Id$ */ #include "squid.h" +#include "ssl/gadgets.h" #include "ssl/crtd_message.h" #if HAVE_CSTDLIB #include #endif #if HAVE_CSTRING #include #endif +#if HAVE_STDEXCEPT +#include +#endif Ssl::CrtdMessage::CrtdMessage() : body_size(0), state(BEFORE_CODE) {} Ssl::CrtdMessage::ParseResult Ssl::CrtdMessage::parse(const char * buffer, size_t len) { char const *current_pos = buffer; while (current_pos != buffer + len && state != END) { switch (state) { case BEFORE_CODE: { if (xisspace(*current_pos)) { current_pos++; break; } if (xisalpha(*current_pos)) { state = CODE; break; } clear(); @@ -156,22 +160,103 @@ std::string param(current_string.c_str(), current_string.c_str() + equal_pos); std::string value(current_string.c_str() + equal_pos + 1); map.insert(std::make_pair(param, value)); } token = strtok(NULL, "\r\n"); } } void Ssl::CrtdMessage::composeBody(CrtdMessage::BodyParams const & map, std::string const & other_part) { body.clear(); for (BodyParams::const_iterator i = map.begin(); i != map.end(); i++) { if (i != map.begin()) body += "\n"; body += i->first + "=" + i->second; } if (!other_part.empty()) body += '\n' + other_part; } + +bool Ssl::CrtdMessage::parseRequest(Ssl::CertificateProperties &certProperties, std::string &error) +{ + Ssl::CrtdMessage::BodyParams map; + std::string certs_part; + parseBody(map, certs_part); + Ssl::CrtdMessage::BodyParams::iterator i = map.find(Ssl::CrtdMessage::param_host); + if (i == map.end()) { + error = "Cannot find \"host\" parameter in request message"; + return false; + } + certProperties.commonName = i->second; + + i = map.find(Ssl::CrtdMessage::param_SetValidAfter); + if (i != map.end() && strcasecmp(i->second.c_str(), "on") == 0) + certProperties.setValidAfter = true; + + i = map.find(Ssl::CrtdMessage::param_SetValidBefore); + if (i != map.end() && strcasecmp(i->second.c_str(), "on") == 0) + certProperties.setValidBefore = true; + + i = map.find(Ssl::CrtdMessage::param_SetCommonName); + if (i != map.end()) { + // use this as Common Name instead of the hostname + // defined with host or Common Name from mimic cert + certProperties.commonName = i->second; + certProperties.setCommonName = true; + } + + i = map.find(Ssl::CrtdMessage::param_Sign); + if (i != map.end()) { + if ((certProperties.signAlgorithm = Ssl::certSignAlgorithmId(i->second.c_str())) == Ssl::algSignEnd) { + error = "Wrong signing algoritm: " + i->second; + return false; + } + } + else + certProperties.signAlgorithm = Ssl::algSignTrusted; + + if (!Ssl::readCertAndPrivateKeyFromMemory(certProperties.signWithX509, certProperties.signWithPkey, certs_part.c_str())) { + error = "Broken signing certificate!"; + return false; + } + + static const std::string CERT_BEGIN_STR("-----BEGIN CERTIFICATE"); + size_t pos; + if ((pos = certs_part.find(CERT_BEGIN_STR)) != std::string::npos) { + pos += CERT_BEGIN_STR.length(); + if ((pos= certs_part.find(CERT_BEGIN_STR, pos)) != std::string::npos) + Ssl::readCertFromMemory(certProperties.mimicCert, certs_part.c_str() + pos); + } + return true; +} + +void Ssl::CrtdMessage::composeRequest(Ssl::CertificateProperties const &certProperties) +{ + body.clear(); + body = Ssl::CrtdMessage::param_host + "=" + certProperties.commonName; + if (certProperties.setCommonName) + body += "\n" + Ssl::CrtdMessage::param_SetCommonName + "=" + certProperties.commonName; + if (certProperties.setValidAfter) + body += "\n" + Ssl::CrtdMessage::param_SetValidAfter + "=on"; + if (certProperties.setValidBefore) + body += "\n" + Ssl::CrtdMessage::param_SetValidBefore + "=on"; + if(certProperties.signAlgorithm != Ssl::algSignEnd) + body += "\n" + Ssl::CrtdMessage::param_Sign + "=" + certSignAlgorithm(certProperties.signAlgorithm); + + std::string certsPart; + if (!Ssl::writeCertAndPrivateKeyToMemory(certProperties.signWithX509, certProperties.signWithPkey, certsPart)) + throw std::runtime_error("Ssl::writeCertAndPrivateKeyToMemory()"); + if (certProperties.mimicCert.get()) { + if (!Ssl::appendCertToMemory(certProperties.mimicCert, certsPart)) + throw std::runtime_error("Ssl::appendCertToMemory()"); + } + body += "\n" + certsPart; +} + const std::string Ssl::CrtdMessage::code_new_certificate("new_certificate"); const std::string Ssl::CrtdMessage::param_host("host"); +const std::string Ssl::CrtdMessage::param_SetValidAfter(Ssl::CertAdaptAlgorithmStr[algSetValidAfter]); +const std::string Ssl::CrtdMessage::param_SetValidBefore(Ssl::CertAdaptAlgorithmStr[algSetValidBefore]); +const std::string Ssl::CrtdMessage::param_SetCommonName(Ssl::CertAdaptAlgorithmStr[algSetCommonName]); +const std::string Ssl::CrtdMessage::param_Sign("Sign"); === modified file 'src/ssl/crtd_message.h' --- src/ssl/crtd_message.h 2010-11-18 08:01:53 +0000 +++ src/ssl/crtd_message.h 2012-06-20 14:27:31 +0000 @@ -1,36 +1,38 @@ /* * $Id$ */ #ifndef SQUID_SSL_CRTD_MESSAGE_H #define SQUID_SSL_CRTD_MESSAGE_H #if HAVE_STRING #include #endif #if HAVE_MAP #include #endif namespace Ssl { +class CertificateProperties; + /** * This class is responsible for composing and parsing messages destined to, or comming * from an ssl_crtd server. Format of these mesages is: * */ class CrtdMessage { public: typedef std::map BodyParams; /// Parse result codes. enum ParseResult { OK, INCOMPLETE, ERROR }; CrtdMessage(); /**Parse buffer of length len \retval OK if parsing completes \retval INCOMPLETE if more data required \retval ERROR if there is an error. @@ -44,43 +46,56 @@ void setCode(std::string const & aCode); ///< Set new request/reply code to compose. std::string compose() const; ///< Compose current (request) code and body to string. /// Reset the class. void clear(); /** *Parse body data which has the form: \verbatim param1=value1 param2=value2 The other multistring part of body. \endverbatim * The parameters of the body stored to map and the remaining part to other_part */ void parseBody(BodyParams & map, std::string & other_part) const; /** *Compose parameters given by map with their values and the other part given by * other_part to body data. The constructed body will have the form: \verbatim param1=value1 param2=value2 The other multistring part of body. \endverbatim */ void composeBody(BodyParams const & map, std::string const & other_part); + + /// orchestrates entire request parsing + bool parseRequest(Ssl::CertificateProperties &, std::string &error); + void composeRequest(Ssl::CertificateProperties const &); // throws + /// String code for "new_certificate" messages static const std::string code_new_certificate; /// Parameter name for passing hostname static const std::string param_host; + /// Parameter name for passing SetValidAfter cert adaptation variable + static const std::string param_SetValidAfter; + /// Parameter name for passing SetValidBefore cert adaptation variable + static const std::string param_SetValidBefore; + /// Parameter name for passing SetCommonName cert adaptation variable + static const std::string param_SetCommonName; + /// Parameter name for passing signing algorithm + static const std::string param_Sign; private: enum ParseState { BEFORE_CODE, CODE, BEFORE_LENGTH, LENGTH, BEFORE_BODY, BODY, END }; size_t body_size; ///< The body size if exist or 0. ParseState state; ///< Parsing state. std::string body; ///< Current body. std::string code; ///< Current response/request code. std::string current_block; ///< Current block buffer. }; } //namespace Ssl #endif // SQUID_SSL_CRTD_MESSAGE_H === modified file 'src/ssl/gadgets.cc' --- src/ssl/gadgets.cc 2012-02-20 18:07:29 +0000 +++ src/ssl/gadgets.cc 2012-06-11 12:52:32 +0000 @@ -1,241 +1,488 @@ /* * $Id$ */ #include "squid.h" #include "ssl/gadgets.h" #if HAVE_OPENSSL_X509V3_H #include #endif -/** - \ingroup ServerProtocolSSLInternal - * Add CN to subject in request. - */ -static bool addCnToRequest(Ssl::X509_REQ_Pointer & request, char const * cn) -{ - // not an Ssl::X509_NAME_Pointer because X509_REQ_get_subject_name() - // returns a pointer to the existing subject name. Nothing to clean here. - X509_NAME *name = X509_REQ_get_subject_name(request.get()); - if (!name) - return false; - - // The second argument of the X509_NAME_add_entry_by_txt declared as - // "char *" on some OS. Use cn_name to avoid compile warnings. - static char cn_name[3] = "CN"; - if (!X509_NAME_add_entry_by_txt(name, cn_name, MBSTRING_ASC, (unsigned char *)cn, -1, -1, 0)) - return false; - - return true; -} - -/** - \ingroup ServerProtocolSSLInternal - * Make request on sign using private key and hostname. - */ -static bool makeRequest(Ssl::X509_REQ_Pointer & request, Ssl::EVP_PKEY_Pointer const & pkey, char const * host) -{ - if (!X509_REQ_set_version(request.get(), 0L)) - return false; - - if (!addCnToRequest(request, host)) - return false; - - if (!X509_REQ_set_pubkey(request.get(), pkey.get())) - return false; - return true; -} - EVP_PKEY * Ssl::createSslPrivateKey() { Ssl::EVP_PKEY_Pointer pkey(EVP_PKEY_new()); if (!pkey) return NULL; Ssl::RSA_Pointer rsa(RSA_generate_key(1024, RSA_F4, NULL, NULL)); if (!rsa) return NULL; if (!EVP_PKEY_assign_RSA(pkey.get(), (rsa.get()))) return NULL; rsa.release(); return pkey.release(); } -X509_REQ * Ssl::createNewX509Request(Ssl::EVP_PKEY_Pointer const & pkey, const char * hostname) -{ - Ssl::X509_REQ_Pointer request(X509_REQ_new()); - - if (!request) - return NULL; - - if (!makeRequest(request, pkey, hostname)) - return NULL; - return request.release(); -} - /** \ingroup ServerProtocolSSLInternal * Set serial random serial number or set random serial number. */ static bool setSerialNumber(ASN1_INTEGER *ai, BIGNUM const* serial) { if (!ai) return false; Ssl::BIGNUM_Pointer bn(BN_new()); if (serial) { bn.reset(BN_dup(serial)); } else { if (!bn) return false; if (!BN_pseudo_rand(bn.get(), 64, 0, 0)) return false; } if (ai && !BN_to_ASN1_INTEGER(bn.get(), ai)) return false; return true; } -X509 * Ssl::signRequest(Ssl::X509_REQ_Pointer const & request, Ssl::X509_Pointer const & x509, Ssl::EVP_PKEY_Pointer const & pkey, ASN1_TIME * timeNotAfter, BIGNUM const * serial) -{ - Ssl::X509_Pointer cert(X509_new()); - if (!cert) - return NULL; - - if (!setSerialNumber(X509_get_serialNumber(cert.get()), serial)) - return NULL; - - if (!X509_set_issuer_name(cert.get(), x509.get() ? X509_get_subject_name(x509.get()) : X509_REQ_get_subject_name(request.get()))) - return NULL; - - if (!X509_gmtime_adj(X509_get_notBefore(cert.get()), (-2)*24*60*60)) - return NULL; - - if (timeNotAfter) { - if (!X509_set_notAfter(cert.get(), timeNotAfter)) - return NULL; - } else if (!X509_gmtime_adj(X509_get_notAfter(cert.get()), 60*60*24*356*3)) - return NULL; - - if (!X509_set_subject_name(cert.get(), X509_REQ_get_subject_name(request.get()))) - return NULL; - - Ssl::EVP_PKEY_Pointer tmppkey(X509_REQ_get_pubkey(request.get())); - - if (!tmppkey || !X509_set_pubkey(cert.get(), tmppkey.get())) - return NULL; - - if (!X509_sign(cert.get(), pkey.get(), EVP_sha1())) - return NULL; - - return cert.release(); -} - bool Ssl::writeCertAndPrivateKeyToMemory(Ssl::X509_Pointer const & cert, Ssl::EVP_PKEY_Pointer const & pkey, std::string & bufferToWrite) { bufferToWrite.clear(); if (!pkey || !cert) return false; BIO_Pointer bio(BIO_new(BIO_s_mem())); if (!bio) return false; if (!PEM_write_bio_X509 (bio.get(), cert.get())) return false; if (!PEM_write_bio_PrivateKey(bio.get(), pkey.get(), NULL, NULL, 0, NULL, NULL)) return false; char *ptr = NULL; long len = BIO_get_mem_data(bio.get(), &ptr); if (!ptr) return false; bufferToWrite = std::string(ptr, len); return true; } +bool Ssl::appendCertToMemory(Ssl::X509_Pointer const & cert, std::string & bufferToWrite) +{ + if (!cert) + return false; + + BIO_Pointer bio(BIO_new(BIO_s_mem())); + if (!bio) + return false; + + if (!PEM_write_bio_X509 (bio.get(), cert.get())) + return false; + + char *ptr = NULL; + long len = BIO_get_mem_data(bio.get(), &ptr); + if (!ptr) + return false; + + if (!bufferToWrite.empty()) + bufferToWrite.append(" "); // add a space... + + bufferToWrite.append(ptr, len); + return true; +} + bool Ssl::writeCertAndPrivateKeyToFile(Ssl::X509_Pointer const & cert, Ssl::EVP_PKEY_Pointer const & pkey, char const * filename) { if (!pkey || !cert) return false; Ssl::BIO_Pointer bio(BIO_new(BIO_s_file_internal())); if (!bio) return false; if (!BIO_write_filename(bio.get(), const_cast(filename))) return false; if (!PEM_write_bio_X509(bio.get(), cert.get())) return false; if (!PEM_write_bio_PrivateKey(bio.get(), pkey.get(), NULL, NULL, 0, NULL, NULL)) return false; return true; } bool Ssl::readCertAndPrivateKeyFromMemory(Ssl::X509_Pointer & cert, Ssl::EVP_PKEY_Pointer & pkey, char const * bufferToRead) { Ssl::BIO_Pointer bio(BIO_new(BIO_s_mem())); BIO_puts(bio.get(), bufferToRead); X509 * certPtr = NULL; cert.reset(PEM_read_bio_X509(bio.get(), &certPtr, 0, 0)); if (!cert) return false; EVP_PKEY * pkeyPtr = NULL; pkey.reset(PEM_read_bio_PrivateKey(bio.get(), &pkeyPtr, 0, 0)); if (!pkey) return false; return true; } -bool Ssl::generateSslCertificateAndPrivateKey(char const *host, Ssl::X509_Pointer const & signedX509, Ssl::EVP_PKEY_Pointer const & signedPkey, Ssl::X509_Pointer & cert, Ssl::EVP_PKEY_Pointer & pkey, BIGNUM const * serial) +bool Ssl::readCertFromMemory(X509_Pointer & cert, char const * bufferToRead) { - pkey.reset(createSslPrivateKey()); - if (!pkey) + Ssl::BIO_Pointer bio(BIO_new(BIO_s_mem())); + BIO_puts(bio.get(), bufferToRead); + + X509 * certPtr = NULL; + cert.reset(PEM_read_bio_X509(bio.get(), &certPtr, 0, 0)); + if (!cert) return false; - Ssl::X509_REQ_Pointer request(createNewX509Request(pkey, host)); - if (!request) + return true; +} + +// According to RFC 5280 (Section A.1), the common name length in a certificate +// can be at most 64 characters +static const size_t MaxCnLen = 64; + +// Replace certs common name with the given +static bool replaceCommonName(Ssl::X509_Pointer & cert, std::string const &rawCn) +{ + std::string cn = rawCn; + + if (cn.length() > MaxCnLen) { + // In the case the length od CN is more than the maximum supported size + // try to use the first upper level domain. + size_t pos = 0; + do { + pos = cn.find('.', pos + 1); + } while(pos != std::string::npos && (cn.length() - pos + 2) > MaxCnLen); + + // If no short domain found or this domain is a toplevel domain + // we failed to find a good cn name. + if (pos == std::string::npos || cn.find('.', pos + 1) == std::string::npos) + return false; + + std::string fixedCn(1, '*'); + fixedCn.append(cn.c_str() + pos); + cn = fixedCn; + } + + // Assume [] surround an IPv6 address and strip them because browsers such + // as Firefox, Chromium, and Safari prefer bare IPv6 addresses in CNs. + if (cn.length() > 2 && *cn.begin() == '[' && *cn.rbegin() == ']') + cn = cn.substr(1, cn.size()-2); + + X509_NAME *name = X509_get_subject_name(cert.get()); + if (!name) return false; + // Remove the CN part: + int loc = X509_NAME_get_index_by_NID(name, NID_commonName, -1); + if (loc >=0) { + X509_NAME_ENTRY *tmp = X509_NAME_get_entry(name, loc); + X509_NAME_delete_entry(name, loc); + X509_NAME_ENTRY_free(tmp); + } + + // Add a new CN + return X509_NAME_add_entry_by_NID(name, NID_commonName, MBSTRING_ASC, + (unsigned char *)(cn.c_str()), -1, -1, 0); +} + +const char *Ssl::CertSignAlgorithmStr[] = { + "signTrusted", + "signUntrusted", + "signSelf", + NULL +}; + +const char *Ssl::CertAdaptAlgorithmStr[] = { + "setValidAfter", + "setValidBefore", + "setCommonName", + NULL +}; + +Ssl::CertificateProperties::CertificateProperties(): + setValidAfter(false), + setValidBefore(false), + setCommonName(false), + signAlgorithm(Ssl::algSignEnd) +{} + +std::string & Ssl::CertificateProperties::dbKey() const +{ + static std::string certKey; + certKey.clear(); + certKey.reserve(4096); + if (mimicCert.get()) { + char buf[1024]; + certKey.append(X509_NAME_oneline(X509_get_subject_name(mimicCert.get()), buf, sizeof(buf))); + } + + if (certKey.empty()) { + certKey.append("/CN=", 4); + certKey.append(commonName); + } - if (signedX509.get() && signedPkey.get()) - cert.reset(signRequest(request, signedX509, signedPkey, X509_get_notAfter(signedX509.get()), serial)); - else - cert.reset(signRequest(request, signedX509, pkey, NULL, serial)); + if (setValidAfter) + certKey.append("+SetValidAfter=on", 17); + + if (setValidBefore) + certKey.append("+SetValidBefore=on", 18); + + if (setCommonName) { + certKey.append("+SetCommonName=", 15); + certKey.append(commonName); + } + + if (signAlgorithm != Ssl::algSignEnd) { + certKey.append("+Sign=", 6); + certKey.append(certSignAlgorithm(signAlgorithm)); + } + + return certKey; +} + +static bool buildCertificate(Ssl::X509_Pointer & cert, Ssl::CertificateProperties const &properties) +{ + // not an Ssl::X509_NAME_Pointer because X509_REQ_get_subject_name() + // returns a pointer to the existing subject name. Nothing to clean here. + if (properties.mimicCert.get()) { + // Leave subject empty if we cannot extract it from true cert. + if (X509_NAME *name = X509_get_subject_name(properties.mimicCert.get())) { + // X509_set_subject_name will call X509_dup for name + X509_set_subject_name(cert.get(), name); + } + } + + if (properties.setCommonName || !properties.mimicCert.get()) { + // In this case the CN of the certificate given by the user + // Ignore errors: it is better to make a certificate with no CN + // than to quit ssl_crtd because we cannot make a certificate. + // Most errors are caused by user input such as huge domain names. + (void)replaceCommonName(cert, properties.commonName); + } + // We should get caCert notBefore and notAfter fields and do not allow + // notBefore/notAfter values from certToMimic before/after notBefore/notAfter + // fields from caCert. + // Currently there is not any way in openssl tollkit to compare two ASN1_TIME + // objects. + ASN1_TIME *aTime = NULL; + if (!properties.setValidBefore && properties.mimicCert.get()) + aTime = X509_get_notBefore(properties.mimicCert.get()); + if (!aTime && properties.signWithX509.get()) + aTime = X509_get_notBefore(properties.signWithX509.get()); + + if (aTime) { + if (!X509_set_notBefore(cert.get(), aTime)) + return false; + } + else if (!X509_gmtime_adj(X509_get_notBefore(cert.get()), (-2)*24*60*60)) + return false; + + aTime = NULL; + if (!properties.setValidAfter && properties.mimicCert.get()) + aTime = X509_get_notAfter(properties.mimicCert.get()); + if (!aTime && properties.signWithX509.get()) + aTime = X509_get_notAfter(properties.signWithX509.get()); + if (aTime) { + if (!X509_set_notAfter(cert.get(), aTime)) + return false; + } else if (!X509_gmtime_adj(X509_get_notAfter(cert.get()), 60*60*24*356*3)) + return false; + + // mimic the alias and possibly subjectAltName + if (properties.mimicCert.get()) { + unsigned char *alStr; + int alLen; + alStr = X509_alias_get0(properties.mimicCert.get(), &alLen); + if (alStr) { + X509_alias_set1(cert.get(), alStr, alLen); + } + + // Mimic subjectAltName unless we used a configured CN: browsers reject + // certificates with CN unrelated to subjectAltNames. + if (!properties.setCommonName) { + int pos=X509_get_ext_by_NID (properties.mimicCert.get(), OBJ_sn2nid("subjectAltName"), -1); + X509_EXTENSION *ext=X509_get_ext(properties.mimicCert.get(), pos); + if (ext) { + X509_add_ext(cert.get(), ext, -1); + /* According the RFC 5280 using extensions requires version 3 + certificate. + Set version value to 2 for version 3 certificates. + */ + X509_set_version(cert.get(), 2); + } + } + } + + return true; +} + +static bool generateFakeSslCertificate(Ssl::X509_Pointer & certToStore, Ssl::EVP_PKEY_Pointer & pkeyToStore, Ssl::CertificateProperties const &properties, Ssl::BIGNUM_Pointer const &serial) +{ + Ssl::EVP_PKEY_Pointer pkey; + // Use signing certificates private key as generated certificate private key + if (properties.signWithPkey.get()) + pkey.resetAndLock(properties.signWithPkey.get()); + else // if not exist generate one + pkey.reset(Ssl::createSslPrivateKey()); + + if (!pkey) + return false; + + Ssl::X509_Pointer cert(X509_new()); if (!cert) return false; + // Set pub key and serial given by the caller + if (!X509_set_pubkey(cert.get(), pkey.get())) + return false; + if (!setSerialNumber(X509_get_serialNumber(cert.get()), serial.get())) + return false; + + // Fill the certificate with the required properties + if (!buildCertificate(cert, properties)) + return false; + + int ret = 0; + // Set issuer name, from CA or our subject name for self signed cert + if (properties.signAlgorithm != Ssl::algSignSelf && properties.signWithX509.get()) + ret = X509_set_issuer_name(cert.get(), X509_get_subject_name(properties.signWithX509.get())); + else // Self signed certificate, set issuer to self + ret = X509_set_issuer_name(cert.get(), X509_get_subject_name(cert.get())); + if (!ret) + return false; + + /*Now sign the request */ + if (properties.signAlgorithm != Ssl::algSignSelf && properties.signWithPkey.get()) + ret = X509_sign(cert.get(), properties.signWithPkey.get(), EVP_sha1()); + else //else sign with self key (self signed request) + ret = X509_sign(cert.get(), pkey.get(), EVP_sha1()); + + if (!ret) + return false; + + certToStore.reset(cert.release()); + pkeyToStore.reset(pkey.release()); return true; } +static BIGNUM *createCertSerial(unsigned char *md, unsigned int n) +{ + + assert(n == 20); //for sha1 n is 20 (for md5 n is 16) + + BIGNUM *serial = NULL; + serial = BN_bin2bn(md, n, NULL); + + // if the serial is "0" set it to '1' + if (BN_is_zero(serial)) + BN_one(serial); + + // serial size does not exceed 20 bytes + assert(BN_num_bits(serial) <= 160); + + // According the RFC 5280, serial is an 20 bytes ASN.1 INTEGER (a signed big integer) + // and the maximum value for X.509 certificate serial number is 2^159-1 and + // the minimum 0. If the first bit of the serial is '1' ( eg 2^160-1), + // will result to a negative integer. + // To handle this, if the produced serial is greater than 2^159-1 + // truncate the last bit + if (BN_is_bit_set(serial, 159)) + BN_clear_bit(serial, 159); + + return serial; +} + +/// Return the SHA1 digest of the DER encoded version of the certificate +/// stored in a BIGNUM +static BIGNUM *x509Digest(Ssl::X509_Pointer const & cert) +{ + unsigned int n; + unsigned char md[EVP_MAX_MD_SIZE]; + + if (!X509_digest(cert.get(),EVP_sha1(),md,&n)) + return NULL; + + return createCertSerial(md, n); +} + +static BIGNUM *x509Pubkeydigest(Ssl::X509_Pointer const & cert) +{ + unsigned int n; + unsigned char md[EVP_MAX_MD_SIZE]; + + if (!X509_pubkey_digest(cert.get(),EVP_sha1(),md,&n)) + return NULL; + + return createCertSerial(md, n); +} + +/// Generate a unique serial number based on a Ssl::CertificateProperties object +/// for a new generated certificate +static bool createSerial(Ssl::BIGNUM_Pointer &serial, Ssl::CertificateProperties const &properties) +{ + Ssl::EVP_PKEY_Pointer fakePkey; + Ssl::X509_Pointer fakeCert; + + serial.reset(x509Pubkeydigest(properties.signWithX509)); + if (!serial.get()) { + serial.reset(BN_new()); + BN_is_zero(serial.get()); + } + + if (!generateFakeSslCertificate(fakeCert, fakePkey, properties, serial)) + return false; + + // The x509Fingerprint return an SHA1 hash. + // both SHA1 hash and maximum serial number size are 20 bytes. + BIGNUM *r = x509Digest(fakeCert); + if (!r) + return false; + + serial.reset(r); + return true; +} + +bool Ssl::generateSslCertificate(Ssl::X509_Pointer & certToStore, Ssl::EVP_PKEY_Pointer & pkeyToStore, Ssl::CertificateProperties const &properties) +{ + Ssl::BIGNUM_Pointer serial; + + if (!createSerial(serial, properties)) + return false; + + return generateFakeSslCertificate(certToStore, pkeyToStore, properties, serial); +} + /** \ingroup ServerProtocolSSLInternal * Read certificate from file. */ static X509 * readSslX509Certificate(char const * certFilename) { if (!certFilename) return NULL; Ssl::BIO_Pointer bio(BIO_new(BIO_s_file_internal())); if (!bio) return NULL; if (!BIO_read_filename(bio.get(), certFilename)) return NULL; X509 *certificate = PEM_read_bio_X509(bio.get(), NULL, NULL, NULL); return certificate; } EVP_PKEY * Ssl::readSslPrivateKey(char const * keyFilename, pem_password_cb *passwd_callback) { if (!keyFilename) @@ -254,20 +501,167 @@ if (keyFilename == NULL) keyFilename = certFilename; pkey.reset(readSslPrivateKey(keyFilename)); cert.reset(readSslX509Certificate(certFilename)); if (!pkey || !cert || !X509_check_private_key(cert.get(), pkey.get())) { pkey.reset(NULL); cert.reset(NULL); } } bool Ssl::sslDateIsInTheFuture(char const * date) { ASN1_UTCTIME tm; tm.flags = 0; tm.type = 23; tm.data = (unsigned char *)date; tm.length = strlen(date); return (X509_cmp_current_time(&tm) > 0); } + +/// Print the time represented by a ASN1_TIME struct to a string using GeneralizedTime format +static bool asn1timeToGeneralizedTimeStr(ASN1_TIME *aTime, char *buf, int bufLen) +{ + // ASN1_Time holds time to UTCTime or GeneralizedTime form. + // UTCTime has the form YYMMDDHHMMSS[Z | [+|-]offset] + // GeneralizedTime has the form YYYYMMDDHHMMSS[Z | [+|-] offset] + + // length should have space for data plus 2 extra bytes for the two extra year fields + // plus the '\0' char. + if ((aTime->length + 3) > bufLen) + return false; + + char *str; + if (aTime->type == V_ASN1_UTCTIME) { + if (aTime->data[0] > '5') { // RFC 2459, section 4.1.2.5.1 + buf[0] = '1'; + buf[1] = '9'; + } else { + buf[0] = '2'; + buf[1] = '0'; + } + str = buf +2; + } + else // if (aTime->type == V_ASN1_GENERALIZEDTIME) + str = buf; + + memcpy(str, aTime->data, aTime->length); + str[aTime->length] = '\0'; + return true; +} + +static int asn1time_cmp(ASN1_TIME *asnTime1, ASN1_TIME *asnTime2) +{ + char strTime1[64], strTime2[64]; + if (!asn1timeToGeneralizedTimeStr(asnTime1, strTime1, sizeof(strTime1))) + return -1; + if (!asn1timeToGeneralizedTimeStr(asnTime2, strTime2, sizeof(strTime2))) + return -1; + + return strcmp(strTime1, strTime2); +} + +bool Ssl::certificateMatchesProperties(X509 *cert, CertificateProperties const &properties) +{ + assert(cert); + + // For non self-signed certificates we have to check if the signing certificate changed + if (properties.signAlgorithm != Ssl::algSignSelf) { + assert(properties.signWithX509.get()); + if (X509_check_issued(properties.signWithX509.get(), cert) != X509_V_OK) + return false; + } + + X509 *cert2 = properties.mimicCert.get(); + // If there is not certificate to mimic stop here + if (!cert2) + return true; + + if (!properties.setCommonName) { + X509_NAME *cert1_name = X509_get_subject_name(cert); + X509_NAME *cert2_name = X509_get_subject_name(cert2); + if (X509_NAME_cmp(cert1_name, cert2_name) != 0) + return false; + } + else if (properties.commonName != CommonHostName(cert)) + return false; + + if (!properties.setValidBefore) { + ASN1_TIME *aTime = X509_get_notBefore(cert); + ASN1_TIME *bTime = X509_get_notBefore(cert2); + if (asn1time_cmp(aTime, bTime) != 0) + return false; + } else if (X509_cmp_current_time(X509_get_notBefore(cert)) >= 0) { + // notBefore does not exist (=0) or it is in the future (>0) + return false; + } + + if (!properties.setValidAfter) { + ASN1_TIME *aTime = X509_get_notAfter(cert); + ASN1_TIME *bTime = X509_get_notAfter(cert2); + if (asn1time_cmp(aTime, bTime) != 0) + return false; + } else if (X509_cmp_current_time(X509_get_notAfter(cert)) <= 0) { + // notAfter does not exist (0) or it is in the past (<0) + return false; + } + + + char *alStr1; + int alLen; + alStr1 = (char *)X509_alias_get0(cert, &alLen); + char *alStr2 = (char *)X509_alias_get0(cert2, &alLen); + if ((!alStr1 && alStr2) || (alStr1 && !alStr2) || + (alStr1 && alStr2 && strcmp(alStr1, alStr2)) != 0) + return false; + + // Compare subjectAltName extension + STACK_OF(GENERAL_NAME) * cert1_altnames; + cert1_altnames = (STACK_OF(GENERAL_NAME)*)X509_get_ext_d2i(cert, NID_subject_alt_name, NULL, NULL); + STACK_OF(GENERAL_NAME) * cert2_altnames; + cert2_altnames = (STACK_OF(GENERAL_NAME)*)X509_get_ext_d2i(cert2, NID_subject_alt_name, NULL, NULL); + bool match = true; + if (cert1_altnames) { + int numalts = sk_GENERAL_NAME_num(cert1_altnames); + for (int i = 0; match && i < numalts; i++) { + const GENERAL_NAME *aName = sk_GENERAL_NAME_value(cert1_altnames, i); + match = sk_GENERAL_NAME_find(cert2_altnames, aName); + } + } + else if (cert2_altnames) + match = false; + + sk_GENERAL_NAME_pop_free(cert1_altnames, GENERAL_NAME_free); + sk_GENERAL_NAME_pop_free(cert2_altnames, GENERAL_NAME_free); + + return match; +} + +static const char *getSubjectEntry(X509 *x509, int nid) +{ + static char name[1024] = ""; // stores common name (CN) + + if (!x509) + return NULL; + + // TODO: What if the entry is a UTF8String? See X509_NAME_get_index_by_NID(3ssl). + const int nameLen = X509_NAME_get_text_by_NID( + X509_get_subject_name(x509), + nid, name, sizeof(name)); + + if (nameLen > 0) + return name; + + return NULL; +} + +const char *Ssl::CommonHostName(X509 *x509) +{ + return getSubjectEntry(x509, NID_commonName); +} + +const char *Ssl::getOrganization(X509 *x509) +{ + return getSubjectEntry(x509, NID_organizationName); +} + === modified file 'src/ssl/gadgets.h' --- src/ssl/gadgets.h 2012-02-20 18:07:29 +0000 +++ src/ssl/gadgets.h 2012-04-05 16:21:41 +0000 @@ -1,144 +1,273 @@ /* * 2009/01/17 */ #ifndef SQUID_SSL_GADGETS_H #define SQUID_SSL_GADGETS_H #include "base/TidyPointer.h" +#include "ssl/crtd_message.h" #if HAVE_OPENSSL_SSL_H #include #endif #if HAVE_OPENSSL_TXT_DB_H #include #endif #if HAVE_STRING #include #endif namespace Ssl { /** \defgroup SslCrtdSslAPI ssl_crtd SSL api. These functions must not depend on Squid runtime code such as debug() because they are used by ssl_crtd. */ +/** + \ingroup SslCrtdSslAPI + * Add SSL locking (a.k.a. reference counting) to TidyPointer + */ +template +class LockingPointer: public TidyPointer +{ +public: + typedef TidyPointer Parent; + + LockingPointer(T *t = NULL): Parent(t) { + } + + void resetAndLock(T *t) { + if (t != this->get()) { + reset(t); + if (t) + CRYPTO_add(&t->references, 1, lock); + } + } +}; + // Macro to be used to define the C++ equivalent function of an extern "C" // function. The C++ function suffixed with the _cpp extension #define CtoCpp1(function, argument) \ extern "C++" inline void function ## _cpp(argument a) { \ function(a); \ } /** \ingroup SslCrtdSslAPI * TidyPointer typedefs for common SSL objects */ CtoCpp1(X509_free, X509 *) -typedef TidyPointer X509_Pointer; +typedef LockingPointer X509_Pointer; CtoCpp1(sk_X509_free, STACK_OF(X509) *) typedef TidyPointer X509_STACK_Pointer; CtoCpp1(EVP_PKEY_free, EVP_PKEY *) -typedef TidyPointer EVP_PKEY_Pointer; +typedef LockingPointer EVP_PKEY_Pointer; CtoCpp1(BN_free, BIGNUM *) typedef TidyPointer BIGNUM_Pointer; CtoCpp1(BIO_free, BIO *) typedef TidyPointer BIO_Pointer; CtoCpp1(ASN1_INTEGER_free, ASN1_INTEGER *) typedef TidyPointer ASN1_INT_Pointer; CtoCpp1(TXT_DB_free, TXT_DB *) typedef TidyPointer TXT_DB_Pointer; CtoCpp1(X509_NAME_free, X509_NAME *) typedef TidyPointer X509_NAME_Pointer; CtoCpp1(RSA_free, RSA *) typedef TidyPointer RSA_Pointer; CtoCpp1(X509_REQ_free, X509_REQ *) typedef TidyPointer X509_REQ_Pointer; CtoCpp1(SSL_CTX_free, SSL_CTX *) typedef TidyPointer SSL_CTX_Pointer; CtoCpp1(SSL_free, SSL *) typedef TidyPointer SSL_Pointer; /** \ingroup SslCrtdSslAPI * Create 1024 bits rsa key. */ EVP_PKEY * createSslPrivateKey(); /** \ingroup SslCrtdSslAPI - * Create request on certificate for a host. + * Write private key and SSL certificate to memory. */ -X509_REQ * createNewX509Request(EVP_PKEY_Pointer const & pkey, const char * hostname); +bool writeCertAndPrivateKeyToMemory(X509_Pointer const & cert, EVP_PKEY_Pointer const & pkey, std::string & bufferToWrite); /** \ingroup SslCrtdSslAPI - * Write private key and SSL certificate to memory. + * Append SSL certificate to bufferToWrite. */ -bool writeCertAndPrivateKeyToMemory(X509_Pointer const & cert, EVP_PKEY_Pointer const & pkey, std::string & bufferToWrite); +bool appendCertToMemory(X509_Pointer const & cert, std::string & bufferToWrite); /** \ingroup SslCrtdSslAPI * Write private key and SSL certificate to file. */ bool writeCertAndPrivateKeyToFile(X509_Pointer const & cert, EVP_PKEY_Pointer const & pkey, char const * filename); /** \ingroup SslCrtdSslAPI * Write private key and SSL certificate to memory. */ bool readCertAndPrivateKeyFromMemory(X509_Pointer & cert, EVP_PKEY_Pointer & pkey, char const * bufferToRead); /** \ingroup SslCrtdSslAPI - * Sign SSL request. - * \param x509 if this param equals NULL, returning certificate will be selfsigned. - * \return X509 Signed certificate. + * Read SSL certificate from memory. */ -X509 * signRequest(X509_REQ_Pointer const & request, X509_Pointer const & x509, EVP_PKEY_Pointer const & pkey, ASN1_TIME * timeNotAfter, BIGNUM const * serial); +bool readCertFromMemory(X509_Pointer & cert, char const * bufferToRead); + +/** + \ingroup SslCrtdSslAPI + * Supported certificate signing algorithms + */ +enum CertSignAlgorithm {algSignTrusted = 0, algSignUntrusted, algSignSelf, algSignEnd}; + +/** + \ingroup SslCrtdSslAPI + * Short names for certificate signing algorithms + */ + +extern const char *CertSignAlgorithmStr[]; + +/** + \ingroup SslCrtdSslAPI + * Return the short name of the signing algorithm "sg" + */ +inline const char *certSignAlgorithm(int sg) +{ + if (sg >=0 && sg < Ssl::algSignEnd) + return Ssl::CertSignAlgorithmStr[sg]; + + return NULL; +} + +/** + \ingroup SslCrtdSslAPI + * Return the id of the signing algorithm "sg" + */ +inline CertSignAlgorithm certSignAlgorithmId(const char *sg) +{ + for (int i = 0; i < algSignEnd && Ssl::CertSignAlgorithmStr[i] != NULL; i++) + if (strcmp(Ssl::CertSignAlgorithmStr[i], sg) == 0) + return (CertSignAlgorithm)i; + + return algSignEnd; +} + +/** + \ingroup SslCrtdSslAPI + * Supported certificate adaptation algorithms + */ +enum CertAdaptAlgorithm {algSetValidAfter = 0, algSetValidBefore, algSetCommonName, algSetEnd}; + +/** + \ingroup SslCrtdSslAPI + * Short names for certificate adaptation algorithms + */ +extern const char *CertAdaptAlgorithmStr[]; + +/** + \ingroup SslCrtdSslAPI + * Return the short name of the adaptation algorithm "alg" + */ +inline const char *sslCertAdaptAlgoritm(int alg) +{ + if (alg >=0 && alg < Ssl::algSetEnd) + return Ssl::CertAdaptAlgorithmStr[alg]; + + return NULL; +} + +/** + \ingroup SslCrtdSslAPI + * Simple struct to pass certificate generation parameters to generateSslCertificate function. + */ +class CertificateProperties { +public: + CertificateProperties(); + X509_Pointer mimicCert; ///< Certificate to mimic + X509_Pointer signWithX509; ///< Certificate to sign the generated request + EVP_PKEY_Pointer signWithPkey; ///< The key of the signing certificate + bool setValidAfter; ///< Do not mimic "Not Valid After" field + bool setValidBefore; ///< Do not mimic "Not Valid Before" field + bool setCommonName; ///< Replace the CN field of the mimicing subject with the given + std::string commonName; ///< A CN to use for the generated certificate + CertSignAlgorithm signAlgorithm; ///< The signing algorithm to use + /// Returns certificate database primary key. New fake certificates + /// purge old fake certificates with the same key. + std::string & dbKey() const; +private: + CertificateProperties(CertificateProperties &); + CertificateProperties &operator =(CertificateProperties const &); +}; /** \ingroup SslCrtdSslAPI * Decide on the kind of certificate and generate a CA- or self-signed one. + * The generated certificate will inherite properties from certToMimic * Return generated certificate and private key in resultX509 and resultPkey * variables. */ -bool generateSslCertificateAndPrivateKey(char const *host, X509_Pointer const & signedX509, EVP_PKEY_Pointer const & signedPkey, X509_Pointer & cert, EVP_PKEY_Pointer & pkey, BIGNUM const* serial); +bool generateSslCertificate(X509_Pointer & cert, EVP_PKEY_Pointer & pkey, CertificateProperties const &properties); /** \ingroup SslCrtdSslAPI * Read private key from file. Make sure that this is not encrypted file. */ EVP_PKEY * readSslPrivateKey(char const * keyFilename, pem_password_cb *passwd_callback = NULL); /** \ingroup SslCrtdSslAPI * Read certificate and private key from files. * \param certFilename name of file with certificate. * \param keyFilename name of file with private key. */ void readCertAndPrivateKeyFromFiles(X509_Pointer & cert, EVP_PKEY_Pointer & pkey, char const * certFilename, char const * keyFilename); /** \ingroup SslCrtdSslAPI * Verify date. Date format it ASN1_UTCTIME. if there is out of date error, * return false. */ bool sslDateIsInTheFuture(char const * date); +/** + \ingroup SslCrtdSslAPI + * Check if the major fields of a certificates matches the properties given by + * a CertficateProperties object + \return true if the certificates matches false otherwise. +*/ +bool certificateMatchesProperties(X509 *peer_cert, CertificateProperties const &properties); + +/** + \ingroup ServerProtocolSSLAPI + * Returns CN from the certificate, suitable for use as a host name. + * Uses static memory to temporary store the extracted name. +*/ +const char *CommonHostName(X509 *x509); + +/** + \ingroup ServerProtocolSSLAPI + * Returns Organization from the certificate. + * Uses static memory to temporary store the extracted name. +*/ +const char *getOrganization(X509 *x509); + } // namespace Ssl #endif // SQUID_SSL_GADGETS_H === modified file 'src/ssl/helper.cc' --- src/ssl/helper.cc 2012-04-25 21:08:51 +0000 +++ src/ssl/helper.cc 2012-06-11 12:52:32 +0000 @@ -11,49 +11,47 @@ Ssl::Helper * Ssl::Helper::GetInstance() { static Ssl::Helper sslHelper; return &sslHelper; } Ssl::Helper::Helper() { } Ssl::Helper::~Helper() { Shutdown(); } void Ssl::Helper::Init() { assert(ssl_crtd == NULL); - bool useSslBump = false; - for (AnyP::PortCfg *s = ::Config.Sockaddr.http; s; s = s->next) { - if (s->sslBump) { - useSslBump = true; - break; - } - } - - if (!useSslBump) + // we need to start ssl_crtd only if some port(s) need to bump SSL + bool found = false; + for (AnyP::PortCfg *s = ::Config.Sockaddr.http; !found && s; s = s->next) + found = s->sslBump; + for (AnyP::PortCfg *s = ::Config.Sockaddr.https; !found && s; s = s->next) + found = s->sslBump; + if (!found) return; ssl_crtd = new helper("ssl_crtd"); ssl_crtd->childs.updateLimits(Ssl::TheConfig.ssl_crtdChildren); ssl_crtd->ipc_type = IPC_STREAM; // The crtd messages may contain the eol ('\n') character. We are // going to use the '\1' char as the end-of-message mark. ssl_crtd->eom = '\1'; assert(ssl_crtd->cmdline == NULL); { char *tmp = xstrdup(Ssl::TheConfig.ssl_crtd); char *tmp_begin = tmp; char * token = NULL; bool db_path_was_found = false; bool block_size_was_found = false; char buffer[20] = "2048"; while ((token = strwordtok(NULL, &tmp))) { wordlistAdd(&ssl_crtd->cmdline, token); if (!strcmp(token, "-b")) block_size_was_found = true; === modified file 'src/ssl/ssl_crtd.cc' --- src/ssl/ssl_crtd.cc 2012-01-20 18:55:04 +0000 +++ src/ssl/ssl_crtd.cc 2012-06-20 14:27:31 +0000 @@ -55,48 +55,43 @@ -b fs_block_size File system block size in bytes. Need for processing natural size of certificate on disk. Default value is 2048 bytes." After running write requests in the next format: There are two kind of request now: new_certificate 14 host=host.dom Create new private key and selfsigned certificate for "host.dom". new_certificate xxx host=host.dom -----BEGIN CERTIFICATE----- ... -----END CERTIFICATE----- -----BEGIN RSA PRIVATE KEY----- ... -----END RSA PRIVATE KEY----- Create new private key and certificate request for "host.dom". Sign new request by received certificate and private key. -usage: ssl_crtd -c -s ssl_store_path\n -n new_serial_number +usage: ssl_crtd -c -s ssl_store_path\n -c Init ssl db directories and exit. - -n new_serial_number HEX serial number to use when initializing db. - The default value of serial number is - the number of seconds since Epoch minus 1200000000 -usage: ssl_crtd -g -s ssl_store_path - -g Show current serial number and exit. \endverbatim */ static const char *const B_KBYTES_STR = "KB"; static const char *const B_MBYTES_STR = "MB"; static const char *const B_GBYTES_STR = "GB"; static const char *const B_BYTES_STR = "B"; /** \ingroup ssl_crtd * Get current time. */ time_t getCurrentTime(void) { struct timeval current_time; #if GETTIMEOFDAY_NO_TZP gettimeofday(¤t_time); #else gettimeofday(¤t_time, NULL); #endif @@ -178,166 +173,149 @@ "\t certificates.\n" "\t-M storage_max_size max size of ssl certificates storage.\n" "\t-b fs_block_size File system block size in bytes. Need for processing\n" "\t natural size of certificate on disk. Default value is\n" "\t 2048 bytes.\n" "\n" "After running write requests in the next format:\n" "\n" "There are two kind of request now:\n" + Ssl::CrtdMessage::code_new_certificate + " " + request_string_size_stream.str() + " " + request_string + "\n" + "\tCreate new private key and selfsigned certificate for \"host.dom\".\n" + Ssl::CrtdMessage::code_new_certificate + " xxx " + request_string + "\n" + "-----BEGIN CERTIFICATE-----\n" "...\n" "-----END CERTIFICATE-----\n" "-----BEGIN RSA PRIVATE KEY-----\n" "...\n" "-----END RSA PRIVATE KEY-----\n" "\tCreate new private key and certificate request for \"host.dom\"\n" "\tSign new request by received certificate and private key.\n" - "usage: ssl_crtd -c -s ssl_store_path -n new_serial_number\n" - "\t-c Init ssl db directories and exit.\n" - "\t-n new_serial_number HEX serial number to use when initializing db.\n" - "\t The default value of serial number is\n" - "\t the number of seconds since Epoch minus 1200000000\n" - "usage: ssl_crtd -g -s ssl_store_path\n" - "\t-g Show current serial number and exit."; + "usage: ssl_crtd -c -s ssl_store_path\n" + "\t-c Init ssl db directories and exit.\n"; std::cerr << help_string << std::endl; } /** \ingroup ssl_crtd * Proccess new request message. */ -static bool proccessNewRequest(Ssl::CrtdMessage const & request_message, std::string const & db_path, size_t max_db_size, size_t fs_block_size) +static bool proccessNewRequest(Ssl::CrtdMessage & request_message, std::string const & db_path, size_t max_db_size, size_t fs_block_size) { - Ssl::CrtdMessage::BodyParams map; - std::string body_part; - request_message.parseBody(map, body_part); - - Ssl::CrtdMessage::BodyParams::iterator i = map.find(Ssl::CrtdMessage::param_host); - if (i == map.end()) - throw std::runtime_error("Cannot find \"" + Ssl::CrtdMessage::param_host + "\" parameter in request message."); - std::string host = i->second; + Ssl::CertificateProperties certProperties; + std::string error; + if (!request_message.parseRequest(certProperties, error)) + throw std::runtime_error("Error while parsing the crtd request: " + error); Ssl::CertificateDb db(db_path, max_db_size, fs_block_size); Ssl::X509_Pointer cert; Ssl::EVP_PKEY_Pointer pkey; - db.find("/CN=" + host, cert, pkey); - - if (!cert || !pkey) { - Ssl::X509_Pointer certToSign; - Ssl::EVP_PKEY_Pointer pkeyToSign; - Ssl::readCertAndPrivateKeyFromMemory(certToSign, pkeyToSign, body_part.c_str()); - - Ssl::BIGNUM_Pointer serial(db.getCurrentSerialNumber()); + std::string &cert_subject = certProperties.dbKey(); + + db.find(cert_subject, cert, pkey); + + if (cert.get()) { + if (!Ssl::certificateMatchesProperties(cert.get(), certProperties)) { + // The certificate changed (renewed or other reason). + // Generete a new one with the updated fields. + cert.reset(NULL); + pkey.reset(NULL); + db.purgeCert(cert_subject); + } + } - if (!Ssl::generateSslCertificateAndPrivateKey(host.c_str(), certToSign, pkeyToSign, cert, pkey, serial.get())) + if (!cert || !pkey) { + if (!Ssl::generateSslCertificate(cert, pkey, certProperties)) throw std::runtime_error("Cannot create ssl certificate or private key."); - if (!db.addCertAndPrivateKey(cert, pkey) && db.IsEnabledDiskStore()) + + if (!db.addCertAndPrivateKey(cert, pkey, cert_subject) && db.IsEnabledDiskStore()) throw std::runtime_error("Cannot add certificate to db."); } std::string bufferToWrite; if (!Ssl::writeCertAndPrivateKeyToMemory(cert, pkey, bufferToWrite)) throw std::runtime_error("Cannot write ssl certificate or/and private key to memory."); Ssl::CrtdMessage response_message; response_message.setCode("OK"); response_message.setBody(bufferToWrite); // Use the '\1' char as end-of-message character std::cout << response_message.compose() << '\1' << std::flush; return true; } /** \ingroup ssl_crtd * This is the external ssl_crtd process. */ int main(int argc, char *argv[]) { try { - int serial = (getCurrentTime() - 1200000000); size_t max_db_size = 0; size_t fs_block_size = 2048; char c; bool create_new_db = false; - bool show_sn = false; std::string db_path; // proccess options. while ((c = getopt(argc, argv, "dcghvs:M:b:n:")) != -1) { switch (c) { case 'd': debug_enabled = 1; break; case 'b': if (!parseBytesOptionValue(&fs_block_size, optarg)) { throw std::runtime_error("Error when parsing -b options value"); } break; case 's': db_path = optarg; break; - case 'n': { - std::stringstream sn_stream(optarg); - sn_stream >> std::hex >> serial; - break; - } case 'M': if (!parseBytesOptionValue(&max_db_size, optarg)) { throw std::runtime_error("Error when parsing -M options value"); } break; case 'v': std::cout << "ssl_crtd version " << VERSION << std::endl; exit(0); break; case 'c': create_new_db = true; break; - case 'g': - show_sn = true; - break; case 'h': usage(); exit(0); default: exit(0); } } if (create_new_db) { std::cout << "Initialization SSL db..." << std::endl; - Ssl::CertificateDb::create(db_path, serial); + Ssl::CertificateDb::create(db_path); std::cout << "Done" << std::endl; exit(0); } - if (show_sn) { - Ssl::CertificateDb db(db_path, 4096, 0); - std::cout << db.getSNString() << std::endl; - exit(0); - } { Ssl::CertificateDb::check(db_path, max_db_size); } // proccess request. for (;;) { char request[HELPER_INPUT_BUFFER]; Ssl::CrtdMessage request_message; Ssl::CrtdMessage::ParseResult parse_result = Ssl::CrtdMessage::INCOMPLETE; while (parse_result == Ssl::CrtdMessage::INCOMPLETE) { if (fgets(request, HELPER_INPUT_BUFFER, stdin) == NULL) return 1; size_t gcount = strlen(request); parse_result = request_message.parse(request, gcount); } if (parse_result == Ssl::CrtdMessage::ERROR) { throw std::runtime_error("Cannot parse request message."); } else if (request_message.getCode() == Ssl::CrtdMessage::code_new_certificate) { proccessNewRequest(request_message, db_path, max_db_size, fs_block_size); === modified file 'src/ssl/support.cc' --- src/ssl/support.cc 2012-05-06 01:29:22 +0000 +++ src/ssl/support.cc 2012-06-20 14:27:31 +0000 @@ -29,40 +29,47 @@ * * You should have received a copy of the GNU General Public License * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. * */ #include "squid-old.h" /* MS Visual Studio Projects are monolithic, so we need the following * #if to exclude the SSL code from compile process when not needed. */ #if USE_SSL #include "fde.h" #include "acl/FilledChecklist.h" #include "ssl/ErrorDetail.h" #include "ssl/support.h" #include "ssl/gadgets.h" +const char *Ssl::BumpModeStr[] = { + "none", + "client-first", + "server-first", + NULL +}; + /** \defgroup ServerProtocolSSLInternal Server-Side SSL Internals \ingroup ServerProtocolSSLAPI */ /// \ingroup ServerProtocolSSLInternal static int ssl_ask_password_cb(char *buf, int size, int rwflag, void *userdata) { FILE *in; int len = 0; char cmdline[1024]; snprintf(cmdline, sizeof(cmdline), "\"%s\" \"%s\"", Config.Program.ssl_password, (const char *)userdata); in = popen(cmdline, "r"); if (fgets(buf, size, in)) len = strlen(buf); @@ -183,103 +190,132 @@ } sk_GENERAL_NAME_pop_free(altnames, GENERAL_NAME_free); } return 0; } static int check_domain( void *check_data, ASN1_STRING *cn_data) { char cn[1024]; const char *server = (const char *)check_data; if (cn_data->length > (int)sizeof(cn) - 1) { return 1; //if does not fit our buffer just ignore } memcpy(cn, cn_data->data, cn_data->length); cn[cn_data->length] = '\0'; debugs(83, 4, "Verifying server domain " << server << " to certificate name/subjectAltName " << cn); return matchDomainName(server, cn[0] == '*' ? cn + 1 : cn); } +bool Ssl::checkX509ServerValidity(X509 *cert, const char *server) +{ + return matchX509CommonNames(cert, (void *)server, check_domain); +} + /// \ingroup ServerProtocolSSLInternal static int ssl_verify_cb(int ok, X509_STORE_CTX * ctx) { // preserve original ctx->error before SSL_ calls can overwrite it Ssl::ssl_error_t error_no = ok ? SSL_ERROR_NONE : ctx->error; char buffer[256] = ""; SSL *ssl = (SSL *)X509_STORE_CTX_get_ex_data(ctx, SSL_get_ex_data_X509_STORE_CTX_idx()); SSL_CTX *sslctx = SSL_get_SSL_CTX(ssl); const char *server = (const char *)SSL_get_ex_data(ssl, ssl_ex_index_server); void *dont_verify_domain = SSL_CTX_get_ex_data(sslctx, ssl_ctx_ex_index_dont_verify_domain); ACLChecklist *check = (ACLChecklist*)SSL_get_ex_data(ssl, ssl_ex_index_cert_error_check); + X509 *peeked_cert = (X509 *)SSL_get_ex_data(ssl, ssl_ex_index_ssl_peeked_cert); X509 *peer_cert = ctx->cert; X509_NAME_oneline(X509_get_subject_name(peer_cert), buffer, sizeof(buffer)); if (ok) { debugs(83, 5, "SSL Certificate signature OK: " << buffer); if (server) { - int found = Ssl::matchX509CommonNames(peer_cert, (void *)server, check_domain); - - if (!found) { + if (!Ssl::checkX509ServerValidity(peer_cert, server)) { debugs(83, 2, "SQUID_X509_V_ERR_DOMAIN_MISMATCH: Certificate " << buffer << " does not match domainname " << server); ok = 0; error_no = SQUID_X509_V_ERR_DOMAIN_MISMATCH; } } } + if (ok && peeked_cert) { + // Check whether the already peeked certificate matches the new one. + if (X509_cmp(peer_cert, peeked_cert) != 0) { + debugs(83, 2, "SQUID_X509_V_ERR_CERT_CHANGE: Certificate " << buffer << " does not match peeked certificate"); + ok = 0; + error_no = SQUID_X509_V_ERR_CERT_CHANGE; + } + } + if (!ok) { + Ssl::Errors *errs = static_cast(SSL_get_ex_data(ssl, ssl_ex_index_ssl_errors)); + if (!errs) { + errs = new Ssl::Errors(error_no); + if (!SSL_set_ex_data(ssl, ssl_ex_index_ssl_errors, (void *)errs)) { + debugs(83, 2, "Failed to set ssl error_no in ssl_verify_cb: Certificate " << buffer); + delete errs; + errs = NULL; + } + } + else // remember another error number + errs->push_back_unique(error_no); + if (const char *err_descr = Ssl::GetErrorDescr(error_no)) debugs(83, 5, err_descr << ": " << buffer); else debugs(83, DBG_IMPORTANT, "SSL unknown certificate error " << error_no << " in " << buffer); if (check) { - Filled(check)->ssl_error = error_no; + ACLFilledChecklist *filledCheck = Filled(check); + assert(!filledCheck->sslErrors); + filledCheck->sslErrors = new Ssl::Errors(error_no); if (check->fastCheck() == ACCESS_ALLOWED) { debugs(83, 3, "bypassing SSL error " << error_no << " in " << buffer); ok = 1; } else { debugs(83, 5, "confirming SSL error " << error_no); } + delete filledCheck->sslErrors; + filledCheck->sslErrors = NULL; } } if (!dont_verify_domain && server) {} if (!ok && !SSL_get_ex_data(ssl, ssl_ex_index_ssl_error_detail) ) { // Find the broken certificate. It may be intermediate. X509 *broken_cert = peer_cert; // reasonable default if search fails // Our SQUID_X509_V_ERR_DOMAIN_MISMATCH implies peer_cert is at fault. if (error_no != SQUID_X509_V_ERR_DOMAIN_MISMATCH) { if (X509 *last_used_cert = X509_STORE_CTX_get_current_cert(ctx)) broken_cert = last_used_cert; } Ssl::ErrorDetail *errDetail = - new Ssl::ErrorDetail(error_no, broken_cert); + new Ssl::ErrorDetail(error_no, peer_cert, broken_cert); if (!SSL_set_ex_data(ssl, ssl_ex_index_ssl_error_detail, errDetail)) { debugs(83, 2, "Failed to set Ssl::ErrorDetail in ssl_verify_cb: Certificate " << buffer); delete errDetail; } } return ok; } /// \ingroup ServerProtocolSSLInternal static struct ssl_option { const char *name; long value; } ssl_options[] = { #if SSL_OP_MICROSOFT_SESS_ID_BUG { @@ -560,79 +596,98 @@ return 0; } // "free" function for SSL_get_ex_new_index("cert_err_check") static void ssl_freeAclChecklist(void *, void *ptr, CRYPTO_EX_DATA *, int, long, void *) { delete static_cast(ptr); // may be NULL } // "free" function for SSL_get_ex_new_index("ssl_error_detail") static void ssl_free_ErrorDetail(void *, void *ptr, CRYPTO_EX_DATA *, int, long, void *) { Ssl::ErrorDetail *errDetail = static_cast (ptr); delete errDetail; } +static void +ssl_free_SslErrors(void *, void *ptr, CRYPTO_EX_DATA *, + int, long, void *) +{ + Ssl::Errors *errs = static_cast (ptr); + delete errs; +} + +// "free" function for X509 certificates +static void +ssl_free_X509(void *, void *ptr, CRYPTO_EX_DATA *, + int, long, void *) +{ + X509 *cert = static_cast (ptr); + X509_free(cert); +} + /// \ingroup ServerProtocolSSLInternal static void ssl_initialize(void) { static int ssl_initialized = 0; if (!ssl_initialized) { ssl_initialized = 1; SSL_load_error_strings(); SSLeay_add_ssl_algorithms(); #if HAVE_OPENSSL_ENGINE_H if (Config.SSL.ssl_engine) { ENGINE *e; if (!(e = ENGINE_by_id(Config.SSL.ssl_engine))) { fatalf("Unable to find SSL engine '%s'\n", Config.SSL.ssl_engine); } if (!ENGINE_set_default(e, ENGINE_METHOD_ALL)) { int ssl_error = ERR_get_error(); fatalf("Failed to initialise SSL engine: %s\n", ERR_error_string(ssl_error, NULL)); } } #else if (Config.SSL.ssl_engine) { fatalf("Your OpenSSL has no SSL engine support\n"); } #endif } ssl_ex_index_server = SSL_get_ex_new_index(0, (void *) "server", NULL, NULL, NULL); ssl_ctx_ex_index_dont_verify_domain = SSL_CTX_get_ex_new_index(0, (void *) "dont_verify_domain", NULL, NULL, NULL); ssl_ex_index_cert_error_check = SSL_get_ex_new_index(0, (void *) "cert_error_check", NULL, &ssl_dupAclChecklist, &ssl_freeAclChecklist); ssl_ex_index_ssl_error_detail = SSL_get_ex_new_index(0, (void *) "ssl_error_detail", NULL, NULL, &ssl_free_ErrorDetail); + ssl_ex_index_ssl_peeked_cert = SSL_get_ex_new_index(0, (void *) "ssl_peeked_cert", NULL, NULL, &ssl_free_X509); + ssl_ex_index_ssl_errors = SSL_get_ex_new_index(0, (void *) "ssl_errors", NULL, NULL, &ssl_free_SslErrors); } /// \ingroup ServerProtocolSSLInternal static int ssl_load_crl(SSL_CTX *sslContext, const char *CRLfile) { X509_STORE *st = SSL_CTX_get_cert_store(sslContext); X509_CRL *crl; BIO *in = BIO_new_file(CRLfile, "r"); int count = 0; if (!in) { debugs(83, 2, "WARNING: Failed to open CRL file '" << CRLfile << "'"); return 0; } while ((crl = PEM_read_bio_X509_CRL(in,NULL,NULL,NULL))) { if (!X509_STORE_add_crl(st, crl)) debugs(83, 2, "WARNING: Failed to add CRL from file '" << CRLfile << "'"); else @@ -651,40 +706,45 @@ int ssl_error; #if OPENSSL_VERSION_NUMBER < 0x00909000L SSL_METHOD *method; #else const SSL_METHOD *method; #endif SSL_CTX *sslContext; long fl = ssl_parse_flags(flags); ssl_initialize(); if (!keyfile) keyfile = certfile; if (!certfile) certfile = keyfile; if (!CAfile) CAfile = clientCA; + if (!certfile) { + debugs(83, DBG_CRITICAL, "ERROR: No certificate file"); + return NULL; + } + switch (version) { case 2: #ifndef OPENSSL_NO_SSL2 debugs(83, 5, "Using SSLv2."); method = SSLv2_server_method(); #else debugs(83, DBG_IMPORTANT, "SSLv2 is not available in this Proxy."); return NULL; #endif break; case 3: debugs(83, 5, "Using SSLv3."); method = SSLv3_server_method(); break; case 4: debugs(83, 5, "Using TLSv1."); method = TLSv1_server_method(); @@ -705,67 +765,68 @@ debugs(83, 5, "Using TLSv1.2"); method = TLSv1_2_server_method(); #else debugs(83, DBG_IMPORTANT, "TLSv1.2 is not available in this Proxy."); return NULL; #endif break; case 1: default: debugs(83, 5, "Using SSLv2/SSLv3."); method = SSLv23_server_method(); break; } sslContext = SSL_CTX_new(method); if (sslContext == NULL) { ssl_error = ERR_get_error(); - fatalf("Failed to allocate SSL context: %s\n", - ERR_error_string(ssl_error, NULL)); + debugs(83, DBG_CRITICAL, "ERROR: Failed to allocate SSL context: " << ERR_error_string(ssl_error, NULL)); + return NULL; } SSL_CTX_set_options(sslContext, ssl_parse_options(options)); if (context && *context) { SSL_CTX_set_session_id_context(sslContext, (const unsigned char *)context, strlen(context)); } if (fl & SSL_FLAG_NO_SESSION_REUSE) { SSL_CTX_set_session_cache_mode(sslContext, SSL_SESS_CACHE_OFF); } if (Config.SSL.unclean_shutdown) { debugs(83, 5, "Enabling quiet SSL shutdowns (RFC violation)."); SSL_CTX_set_quiet_shutdown(sslContext, 1); } if (cipher) { debugs(83, 5, "Using chiper suite " << cipher << "."); if (!SSL_CTX_set_cipher_list(sslContext, cipher)) { ssl_error = ERR_get_error(); - fatalf("Failed to set SSL cipher suite '%s': %s\n", - cipher, ERR_error_string(ssl_error, NULL)); + debugs(83, DBG_CRITICAL, "ERROR: Failed to set SSL cipher suite '" << cipher << "': " << ERR_error_string(ssl_error, NULL)); + SSL_CTX_free(sslContext); + return NULL; } } debugs(83, DBG_IMPORTANT, "Using certificate in " << certfile); if (!SSL_CTX_use_certificate_chain_file(sslContext, certfile)) { ssl_error = ERR_get_error(); debugs(83, DBG_CRITICAL, "ERROR: Failed to acquire SSL certificate '" << certfile << "': " << ERR_error_string(ssl_error, NULL)); SSL_CTX_free(sslContext); return NULL; } debugs(83, DBG_IMPORTANT, "Using private key in " << keyfile); ssl_ask_password(sslContext, keyfile); if (!SSL_CTX_use_PrivateKey_file(sslContext, keyfile, SSL_FILETYPE_PEM)) { ssl_error = ERR_get_error(); debugs(83, DBG_CRITICAL, "ERROR: Failed to acquire SSL private key '" << keyfile << "': " << ERR_error_string(ssl_error, NULL)); SSL_CTX_free(sslContext); return NULL; @@ -1244,65 +1305,68 @@ return NULL; if (!SSL_CTX_use_PrivateKey(sslContext.get(), pkey.get())) return NULL; return sslContext.release(); } SSL_CTX * Ssl::generateSslContextUsingPkeyAndCertFromMemory(const char * data) { Ssl::X509_Pointer cert; Ssl::EVP_PKEY_Pointer pkey; if (!readCertAndPrivateKeyFromMemory(cert, pkey, data)) return NULL; if (!cert || !pkey) return NULL; return createSSLContext(cert, pkey); } -SSL_CTX * Ssl::generateSslContext(char const *host, Ssl::X509_Pointer const & signedX509, Ssl::EVP_PKEY_Pointer const & signedPkey) +SSL_CTX * Ssl::generateSslContext(CertificateProperties const &properties) { Ssl::X509_Pointer cert; Ssl::EVP_PKEY_Pointer pkey; - if (!generateSslCertificateAndPrivateKey(host, signedX509, signedPkey, cert, pkey, NULL)) { + if (!generateSslCertificate(cert, pkey, properties)) return NULL; - } + if (!cert) return NULL; if (!pkey) return NULL; return createSSLContext(cert, pkey); } -bool Ssl::verifySslCertificateDate(SSL_CTX * sslContext) +bool Ssl::verifySslCertificate(SSL_CTX * sslContext, CertificateProperties const &properties) { // Temporary ssl for getting X509 certificate from SSL_CTX. Ssl::SSL_Pointer ssl(SSL_new(sslContext)); X509 * cert = SSL_get_certificate(ssl.get()); ASN1_TIME * time_notBefore = X509_get_notBefore(cert); ASN1_TIME * time_notAfter = X509_get_notAfter(cert); bool ret = (X509_cmp_current_time(time_notBefore) < 0 && X509_cmp_current_time(time_notAfter) > 0); - return ret; + if (!ret) + return false; + + return certificateMatchesProperties(cert, properties); } bool Ssl::setClientSNI(SSL *ssl, const char *fqdn) { //The SSL_CTRL_SET_TLSEXT_HOSTNAME is a openssl macro which indicates // if the TLS servername extension (SNI) is enabled in openssl library. #if defined(SSL_CTRL_SET_TLSEXT_HOSTNAME) if (!SSL_set_tlsext_host_name(ssl, fqdn)) { const int ssl_error = ERR_get_error(); debugs(83, 3, "WARNING: unable to set TLS servername extension (SNI): " << ERR_error_string(ssl_error, NULL) << "\n"); return false; } return true; #else debugs(83, 7, "no support for TLS servername extension (SNI)\n"); return false; #endif } @@ -1359,21 +1423,46 @@ return certificate; } void Ssl::readCertChainAndPrivateKeyFromFiles(X509_Pointer & cert, EVP_PKEY_Pointer & pkey, X509_STACK_Pointer & chain, char const * certFilename, char const * keyFilename) { if (keyFilename == NULL) keyFilename = certFilename; if (!chain) chain.reset(sk_X509_new_null()); if (!chain) debugs(83, DBG_IMPORTANT, "WARNING: unable to allocate memory for cert chain"); pkey.reset(readSslPrivateKey(keyFilename, ssl_ask_password_cb)); cert.reset(readSslX509CertificatesChain(certFilename, chain.get())); if (!pkey || !cert || !X509_check_private_key(cert.get(), pkey.get())) { pkey.reset(NULL); cert.reset(NULL); } } +bool Ssl::generateUntrustedCert(X509_Pointer &untrustedCert, EVP_PKEY_Pointer &untrustedPkey, X509_Pointer const &cert, EVP_PKEY_Pointer const & pkey) +{ + // Generate the self-signed certificate, using a hard-coded subject prefix + Ssl::CertificateProperties certProperties; + if (const char *cn = CommonHostName(cert.get())) { + certProperties.commonName = "Not trusted by \""; + certProperties.commonName += cn; + certProperties.commonName += "\""; + } + else if (const char *org = getOrganization(cert.get())) { + certProperties.commonName = "Not trusted by \""; + certProperties.commonName += org; + certProperties.commonName += "\""; + } + else + certProperties.commonName = "Not trusted"; + certProperties.setCommonName = true; + // O, OU, and other CA subject fields will be mimicked + // Expiration date and other common properties will be mimicked + certProperties.signAlgorithm = Ssl::algSignSelf; + certProperties.signWithPkey.resetAndLock(pkey.get()); + certProperties.mimicCert.resetAndLock(cert.get()); + return Ssl::generateSslCertificate(untrustedCert, untrustedPkey, certProperties); +} + #endif /* USE_SSL */ === modified file 'src/ssl/support.h' --- src/ssl/support.h 2011-11-22 12:00:59 +0000 +++ src/ssl/support.h 2012-06-20 14:27:31 +0000 @@ -18,157 +18,200 @@ * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by * the Free Software Foundation; either version 2 of the License, or * (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA. * */ #ifndef SQUID_SSL_SUPPORT_H #define SQUID_SSL_SUPPORT_H +#include "CbDataList.h" #include "ssl/gadgets.h" #if HAVE_OPENSSL_SSL_H #include #endif #if HAVE_OPENSSL_X509V3_H #include #endif #if HAVE_OPENSSL_ERR_H #include #endif #if HAVE_OPENSSL_ENGINE_H #include #endif /** \defgroup ServerProtocolSSLAPI Server-Side SSL API \ingroup ServerProtocol */ // Custom SSL errors; assumes all official errors are positive +#define SQUID_X509_V_ERR_CERT_CHANGE -3 #define SQUID_ERR_SSL_HANDSHAKE -2 #define SQUID_X509_V_ERR_DOMAIN_MISMATCH -1 // All SSL errors range: from smallest (negative) custom to largest SSL error -#define SQUID_SSL_ERROR_MIN SQUID_ERR_SSL_HANDSHAKE +#define SQUID_SSL_ERROR_MIN SQUID_X509_V_ERR_CERT_CHANGE #define SQUID_SSL_ERROR_MAX INT_MAX namespace Ssl { /// Squid defined error code (<0), an error code returned by SSL X509 api, or SSL_ERROR_NONE typedef int ssl_error_t; + +typedef CbDataList Errors; + } //namespace Ssl /// \ingroup ServerProtocolSSLAPI SSL_CTX *sslCreateServerContext(const char *certfile, const char *keyfile, int version, const char *cipher, const char *options, const char *flags, const char *clientCA, const char *CAfile, const char *CApath, const char *CRLfile, const char *dhpath, const char *context); /// \ingroup ServerProtocolSSLAPI SSL_CTX *sslCreateClientContext(const char *certfile, const char *keyfile, int version, const char *cipher, const char *options, const char *flags, const char *CAfile, const char *CApath, const char *CRLfile); /// \ingroup ServerProtocolSSLAPI int ssl_read_method(int, char *, int); /// \ingroup ServerProtocolSSLAPI int ssl_write_method(int, const char *, int); /// \ingroup ServerProtocolSSLAPI void ssl_shutdown_method(SSL *ssl); /// \ingroup ServerProtocolSSLAPI const char *sslGetUserEmail(SSL *ssl); /// \ingroup ServerProtocolSSLAPI typedef char const *SSLGETATTRIBUTE(SSL *, const char *); /// \ingroup ServerProtocolSSLAPI SSLGETATTRIBUTE sslGetUserAttribute; /// \ingroup ServerProtocolSSLAPI SSLGETATTRIBUTE sslGetCAAttribute; /// \ingroup ServerProtocolSSLAPI const char *sslGetUserCertificatePEM(SSL *ssl); /// \ingroup ServerProtocolSSLAPI const char *sslGetUserCertificateChainPEM(SSL *ssl); namespace Ssl { /** \ingroup ServerProtocolSSLAPI + * Supported ssl-bump modes + */ +enum BumpMode {bumpNone = 0, bumpClientFirst, bumpServerFirst, bumpEnd}; + +/** + \ingroup ServerProtocolSSLAPI + * Short names for ssl-bump modes + */ +extern const char *BumpModeStr[]; + +/** + \ingroup ServerProtocolSSLAPI + * Return the short name of the ssl-bump mode "bm" + */ +inline const char *bumpMode(int bm) +{ + return (0 <= bm && bm < Ssl::bumpEnd) ? Ssl::BumpModeStr[bm] : NULL; +} + +/** + \ingroup ServerProtocolSSLAPI + * Generate a certificate to be used as untrusted signing certificate, based on a trusted CA +*/ +bool generateUntrustedCert(X509_Pointer & untrustedCert, EVP_PKEY_Pointer & untrustedPkey, X509_Pointer const & cert, EVP_PKEY_Pointer const & pkey); + +/** + \ingroup ServerProtocolSSLAPI * Decide on the kind of certificate and generate a CA- or self-signed one */ -SSL_CTX *generateSslContext(char const *host, Ssl::X509_Pointer const & signedX509, Ssl::EVP_PKEY_Pointer const & signedPkey); +SSL_CTX * generateSslContext(CertificateProperties const &properties); /** \ingroup ServerProtocolSSLAPI - * Check date of certificate signature. If there is out of date error fucntion - * returns false, true otherwise. + * Check if the certificate of the given context is still valid + \param sslContext The context to check + \param properties Check if the context certificate matches the given properties + \return true if the contexts certificate is valid, false otherwise */ -bool verifySslCertificateDate(SSL_CTX * sslContext); +bool verifySslCertificate(SSL_CTX * sslContext, CertificateProperties const &properties); /** \ingroup ServerProtocolSSLAPI * Read private key and certificate from memory and generate SSL context * using their. */ SSL_CTX * generateSslContextUsingPkeyAndCertFromMemory(const char * data); /** \ingroup ServerProtocolSSLAPI * Adds the certificates in certList to the certificate chain of the SSL context */ void addChainToSslContext(SSL_CTX *sslContext, STACK_OF(X509) *certList); /** \ingroup ServerProtocolSSLAPI * Read certificate, private key and any certificates which must be chained from files. * See also: Ssl::readCertAndPrivateKeyFromFiles function, defined in gadgets.h * \param certFilename name of file with certificate and certificates which must be chainned. * \param keyFilename name of file with private key. */ void readCertChainAndPrivateKeyFromFiles(X509_Pointer & cert, EVP_PKEY_Pointer & pkey, X509_STACK_Pointer & chain, char const * certFilename, char const * keyFilename); /** \ingroup ServerProtocolSSLAPI * Iterates over the X509 common and alternate names and to see if matches with given data * using the check_func. \param peer_cert The X509 cert to check \param check_data The data with which the X509 CNs compared \param check_func The function used to match X509 CNs. The CN data passed as ASN1_STRING data \return 1 if any of the certificate CN matches, 0 if none matches. */ int matchX509CommonNames(X509 *peer_cert, void *check_data, int (*check_func)(void *check_data, ASN1_STRING *cn_data)); /** \ingroup ServerProtocolSSLAPI + * Check if the certificate is valid for a server + \param cert The X509 cert to check. + \param server The server name. + \return true if the certificate is valid for the server or false otherwise. + */ +bool checkX509ServerValidity(X509 *cert, const char *server); + +/** + \ingroup ServerProtocolSSLAPI * Convert a given ASN1_TIME to a string form. \param tm the time in ASN1_TIME form \param buf the buffer to write the output \param len write at most len bytes \return The number of bytes written */ int asn1timeToString(ASN1_TIME *tm, char *buf, int len); /** \ingroup ServerProtocolSSLAPI * Sets the hostname for the Server Name Indication (SNI) TLS extension * if supported by the used openssl toolkit. \return true if SNI set false otherwise */ bool setClientSNI(SSL *ssl, const char *fqdn); } //namespace Ssl #if _SQUID_MSWIN_ #if defined(__cplusplus) === modified file 'src/structs.h' --- src/structs.h 2012-04-26 01:04:17 +0000 +++ src/structs.h 2012-05-12 16:02:59 +0000 @@ -610,40 +610,42 @@ } warnings; char *store_dir_select_algorithm; int sleep_after_fork; /* microseconds */ time_t minimum_expiry_time; /* seconds */ external_acl *externalAclHelperList; #if USE_SSL struct { char *cert; char *key; int version; char *options; char *cipher; char *cafile; char *capath; char *crlfile; char *flags; acl_access *cert_error; SSL_CTX *sslContext; + sslproxy_cert_sign *cert_sign; + sslproxy_cert_adapt *cert_adapt; } ssl_client; #endif char *accept_filter; int umask; int max_filedescriptors; int workers; CpuAffinityMap *cpuAffinityMap; #if USE_LOADABLE_MODULES wordlist *loadable_module_names; #endif int client_ip_max_connections; struct { int v4_first; ///< Place IPv4 first in the order of DNS results. ssize_t packet_max; ///< maximum size EDNS advertised for DNS replies. } dns; }; @@ -946,85 +948,85 @@ int n_peers; }; struct _iostats { enum { histSize = 16 }; struct { int reads; int reads_deferred; int read_hist[histSize]; int writes; int write_hist[histSize]; } Http, Ftp, Gopher; }; struct request_flags { - request_flags(): range(0),nocache(0),ims(0),auth(0),cachable(0),hierarchical(0),loopdetect(0),proxy_keepalive(0),proxying(0),refresh(0),redirected(0),need_validation(0),fail_on_validation_err(0),stale_if_hit(0),accelerated(0),ignore_cc(0),intercepted(0), - hostVerified(0), - spoof_client_ip(0),internal(0),internalclient(0),must_keepalive(0),chunked_reply(0),stream_error(0),sslBumped(0),destinationIPLookedUp_(0) { + request_flags(): range(0),nocache(0),ims(0),auth(0),cachable(0),hierarchical(0),loopdetect(0),proxy_keepalive(0),proxying(0),refresh(0),redirected(0),need_validation(0),fail_on_validation_err(0),stale_if_hit(0),accelerated(0),ignore_cc(0),intercepted(0),hostVerified(0),spoof_client_ip(0),internal(0),internalclient(0),must_keepalive(0),pinned(0),canRePin(0),chunked_reply(0),stream_error(0),sslPeek(0),sslBumped(0),destinationIPLookedUp_(0) { #if USE_HTTP_VIOLATIONS nocache_hack = 0; #endif #if FOLLOW_X_FORWARDED_FOR done_follow_x_forwarded_for = 0; #endif /* FOLLOW_X_FORWARDED_FOR */ } unsigned int range:1; unsigned int nocache:1; ///< whether the response to this request may be READ from cache unsigned int ims:1; unsigned int auth:1; unsigned int cachable:1; ///< whether the response to thie request may be stored in the cache unsigned int hierarchical:1; unsigned int loopdetect:1; unsigned int proxy_keepalive:1; unsigned int proxying: 1; /* this should be killed, also in httpstateflags */ unsigned int refresh:1; unsigned int redirected:1; unsigned int need_validation:1; unsigned int fail_on_validation_err:1; ///< whether we should fail if validation fails unsigned int stale_if_hit:1; ///< reply is stale if it is a hit #if USE_HTTP_VIOLATIONS unsigned int nocache_hack:1; /* for changing/ignoring no-cache requests */ #endif unsigned int accelerated:1; unsigned int ignore_cc:1; unsigned int intercepted:1; ///< intercepted request unsigned int hostVerified:1; ///< whether the Host: header passed verification unsigned int spoof_client_ip:1; /**< spoof client ip if possible */ unsigned int internal:1; unsigned int internalclient:1; unsigned int must_keepalive:1; unsigned int connection_auth:1; /** Request wants connection oriented auth */ unsigned int connection_auth_disabled:1; /** Connection oriented auth can not be supported */ unsigned int connection_proxy_auth:1; /** Request wants connection oriented auth */ unsigned int pinned:1; /* Request sent on a pinned connection */ + unsigned int canRePin:1; ///< OK to reopen a failed pinned connection unsigned int auth_sent:1; /* Authentication forwarded */ unsigned int no_direct:1; /* Deny direct forwarding unless overriden by always_direct. Used in accelerator mode */ unsigned int chunked_reply:1; /**< Reply with chunked transfer encoding */ unsigned int stream_error:1; /**< Whether stream error has occured */ + unsigned int sslPeek:1; ///< internal ssl-bump request to get server cert unsigned int sslBumped:1; /**< ssl-bumped request*/ // When adding new flags, please update cloneAdaptationImmune() as needed. bool resetTCP() const; void setResetTCP(); void clearResetTCP(); void destinationIPLookupCompleted(); bool destinationIPLookedUp() const; // returns a partial copy of the flags that includes only those flags // that are safe for a related (e.g., ICAP-adapted) request to inherit request_flags cloneAdaptationImmune() const; #if FOLLOW_X_FORWARDED_FOR unsigned int done_follow_x_forwarded_for; #endif /* FOLLOW_X_FORWARDED_FOR */ private: unsigned int reset_tcp:1; @@ -1078,34 +1080,49 @@ int capacity; /* expected maximum for .count, not a hard limit */ int bits_per_entry; /* number of bits allocated for each entry from capacity */ int count; /* number of digested entries */ int del_count; /* number of deletions performed so far */ }; struct _store_rebuild_data { int objcount; /* # objects successfully reloaded */ int expcount; /* # objects expired */ int scancount; /* # entries scanned or read from state file */ int clashcount; /* # swapfile clashes avoided */ int dupcount; /* # duplicates purged */ int cancelcount; /* # SWAP_LOG_DEL objects purged */ int invalid; /* # bad lines */ int badflags; /* # bad e->flags */ int bad_log_op; int zero_object_sz; }; +#if USE_SSL +struct _sslproxy_cert_sign { + int alg; + ACLList *aclList; + sslproxy_cert_sign *next; +}; + +struct _sslproxy_cert_adapt { + int alg; + char *param; + ACLList *aclList; + sslproxy_cert_adapt *next; +}; +#endif + class Logfile; #include "format/Format.h" #include "log/Formats.h" struct _customlog { char *filename; ACLList *aclList; Format::Format *logFormat; Logfile *logfile; customlog *next; Log::Format::log_type type; }; #endif /* SQUID_STRUCTS_H */ === modified file 'src/tunnel.cc' --- src/tunnel.cc 2012-01-20 18:55:04 +0000 +++ src/tunnel.cc 2012-06-12 17:35:44 +0000 @@ -464,75 +464,89 @@ cbdataInternalUnlock(tunnelState); } void TunnelStateData::Connection::closeIfOpen() { if (Comm::IsConnOpen(conn)) conn->close(); } void TunnelStateData::copyRead(Connection &from, IOCB *completion) { assert(from.len == 0); AsyncCall::Pointer call = commCbCall(5,4, "TunnelBlindCopyReadHandler", CommIoCbPtrFun(completion, this)); comm_read(from.conn, from.buf, from.bytesWanted(1, SQUID_TCP_SO_RCVBUF), call); } /** + * Set the HTTP status for this request and sets the read handlers for client + * and server side connections. + */ +static void +tunnelStartShoveling(TunnelStateData *tunnelState) +{ + *tunnelState->status_ptr = HTTP_OK; + if (cbdataReferenceValid(tunnelState)) { + tunnelState->copyRead(tunnelState->server, TunnelStateData::ReadServer); + tunnelState->copyRead(tunnelState->client, TunnelStateData::ReadClient); + } +} + +/** * All the pieces we need to write to client and/or server connection * have been written. - * - Set the HTTP status for this request. - * - Start the blind pump. + * Call the tunnelStartShoveling to start the blind pump. */ static void tunnelConnectedWriteDone(const Comm::ConnectionPointer &conn, char *buf, size_t size, comm_err_t flag, int xerrno, void *data) { TunnelStateData *tunnelState = (TunnelStateData *)data; debugs(26, 3, HERE << conn << ", flag=" << flag); if (flag != COMM_OK) { *tunnelState->status_ptr = HTTP_INTERNAL_SERVER_ERROR; tunnelErrorComplete(conn->fd, data, 0); return; } - *tunnelState->status_ptr = HTTP_OK; - if (cbdataReferenceValid(tunnelState)) { - tunnelState->copyRead(tunnelState->server, TunnelStateData::ReadServer); - tunnelState->copyRead(tunnelState->client, TunnelStateData::ReadClient); - } + tunnelStartShoveling(tunnelState); } /* * handle the write completion from a proxy request to an upstream origin */ static void tunnelConnected(const Comm::ConnectionPointer &server, void *data) { TunnelStateData *tunnelState = (TunnelStateData *)data; debugs(26, 3, HERE << server << ", tunnelState=" << tunnelState); - AsyncCall::Pointer call = commCbCall(5,5, "tunnelConnectedWriteDone", - CommIoCbPtrFun(tunnelConnectedWriteDone, tunnelState)); - Comm::Write(tunnelState->client.conn, conn_established, strlen(conn_established), call, NULL); + + if (tunnelState->request && (tunnelState->request->flags.spoof_client_ip || tunnelState->request->flags.intercepted)) + tunnelStartShoveling(tunnelState); // ssl-bumped connection, be quiet + else { + AsyncCall::Pointer call = commCbCall(5,5, "tunnelConnectedWriteDone", + CommIoCbPtrFun(tunnelConnectedWriteDone, tunnelState)); + Comm::Write(tunnelState->client.conn, conn_established, strlen(conn_established), call, NULL); + } } static void tunnelErrorComplete(int fd/*const Comm::ConnectionPointer &*/, void *data, size_t) { TunnelStateData *tunnelState = (TunnelStateData *)data; debugs(26, 3, HERE << "FD " << fd); assert(tunnelState != NULL); /* temporary lock to save our own feets (comm_close -> tunnelClientClosed -> Free) */ cbdataInternalLock(tunnelState); if (Comm::IsConnOpen(tunnelState->client.conn)) tunnelState->client.conn->close(); if (Comm::IsConnOpen(tunnelState->server.conn)) tunnelState->server.conn->close(); cbdataInternalUnlock(tunnelState); } === modified file 'src/typedefs.h' --- src/typedefs.h 2012-03-04 22:24:58 +0000 +++ src/typedefs.h 2012-03-08 12:41:28 +0000 @@ -74,40 +74,46 @@ typedef struct _http_state_flags http_state_flags; typedef struct _header_mangler header_mangler; typedef struct _cachemgr_passwd cachemgr_passwd; typedef struct _refresh_t refresh_t; typedef struct _CommWriteStateData CommWriteStateData; typedef struct _storeSwapLogData storeSwapLogData; typedef struct _CacheDigest CacheDigest; typedef struct _Version Version; typedef struct _link_list link_list; typedef struct _customlog customlog; +#if USE_SSL +typedef struct _sslproxy_cert_sign sslproxy_cert_sign; + +typedef struct _sslproxy_cert_adapt sslproxy_cert_adapt; +#endif + #if SQUID_SNMP typedef variable_list *(oid_ParseFn) (variable_list *, snint *); typedef struct _snmp_request_t snmp_request_t; #endif typedef void FREE(void *); typedef void CBDUNL(void *); typedef void FOCB(void *, int fd, int errcode); typedef void PF(int, void *); /* disk.c / diskd.c callback typedefs */ typedef void DRCB(int, const char *buf, int size, int errflag, void *data); /* Disk read CB */ typedef void DWCB(int, int, size_t, void *); /* disk write CB */ typedef void DOCB(int, int errflag, void *data); /* disk open CB */ typedef void DCCB(int, int errflag, void *data); /* disk close CB */ typedef void DUCB(int errflag, void *data); /* disk unlink CB */ typedef void DTCB(int errflag, void *data); /* disk trunc CB */ === modified file 'src/whois.cc' --- src/whois.cc 2012-01-20 18:55:04 +0000 +++ src/whois.cc 2012-04-18 07:22:36 +0000 @@ -142,41 +142,41 @@ void WhoisState::readReply(const Comm::ConnectionPointer &conn, char *aBuffer, size_t aBufferLength, comm_err_t flag, int xerrno) { /* Bail out early on COMM_ERR_CLOSING - close handlers will tidy up for us */ if (flag == COMM_ERR_CLOSING) return; aBuffer[aBufferLength] = '\0'; debugs(75, 3, HERE << conn << " read " << aBufferLength << " bytes"); debugs(75, 5, "{" << aBuffer << "}"); if (flag != COMM_OK) { debugs(50, 2, HERE << conn << ": read failure: " << xstrerror() << "."); if (ignoreErrno(errno)) { AsyncCall::Pointer call = commCbCall(5,4, "whoisReadReply", CommIoCbPtrFun(whoisReadReply, this)); comm_read(conn, aBuffer, BUFSIZ, call); } else { ErrorState *err = new ErrorState(ERR_READ_ERROR, HTTP_INTERNAL_SERVER_ERROR, fwd->request); - err->xerrno = errno; + err->xerrno = xerrno; fwd->fail(err); conn->close(); } return; } if (aBufferLength > 0) { if (!dataWritten) setReplyToOK(entry); kb_incr(&(statCounter.server.all.kbytes_in), aBufferLength); kb_incr(&(statCounter.server.http.kbytes_in), aBufferLength); /* No range support, we always grab it all */ dataWritten = true; entry->append(aBuffer, aBufferLength); entry->flush(); AsyncCall::Pointer call = commCbCall(5,4, "whoisReadReply", CommIoCbPtrFun(whoisReadReply, this));